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Foreword 


Timothy O’Shea 


This book was produced to mark the Computers and Learning research 
group’s fortieth year. The CALRG, as it is known, has a special place in the his- 
tory of The Open University. When it was established back in 1978, The Open 
University was still young, and only in its eighth year of the innovative teaching 
of students at a distance. 

From the outset there was a recognition of the potential of the computer to 
assist and enrich our students’ experience of being distance learners. In1975 we 
offered remedial tutorial Computer Assisted Learning (with feedback) via ter- 
minals in study centres, simulations at science summer schools and day schools 
and a specially designed programming language to help psychology students 
construct cognitive models. 

The folk who came together in 1978 to form this very interdisciplinary 
research group understood that we had to make effective use of these newly 
developing technologies and we were very ambitious. We wanted to develop a 
research group that would cross faculty boundaries and be world leading. The 
Open University is the twentieth century’s greatest educational experiment. As 
such the group saw it could be considered as a test bed with which to explore 
all the possible ways that the quality of student learning could be impacted by 
the judicious and imaginative use of computers in our courses. This proved to 
be a very wise endeavour! 


xvi Foreword 


The group is based in the Institute of Educational Technology with founder 
members including academics from Psychology, and the Education, Mathe- 
matics and Science Faculties, the then Student Computing Services and the 
staff based in regional offices. The original vision of a group with a strong inter- 
national reputation for research excellence and with a mission to help deliver 
better learning experiences for students, has persisted and flourished. 

The group understood that a large cohort of PhD students would be vital for 
the development of a research community so I offer a special acknowledgment 
both to current PhD students and to all the past PhD students who are the 
lifeblood of the group. 

Thanks to the PhD programme and the work of various research staff over 
the years, the group has had an enviable record of external funding for research, 
well cited publications and well documented contributions to the success of 
The Open University in the various UK research assessment exercises. 

As the chapters of this book demonstrate, the group is a key influencer in 
the research and development of educational technology worldwide. The mem- 
bers of the group have contributed to theory development, been technological 
innovators and conducted very large scale teaching experiments. The group's 
unique position comes from supporting and being embedded in a University 
whose raison detre derives from the application of educational technology in 
distance education. 


CHAPTER | 


Introduction 


Ann Jones, Eileen Scanlon and Rebecca Ferguson 


This chapter forms the introduction to the book, “Educational Visions” and 
describes the nature of the book. It presents the four principles or visions which 
inform the work of the Computers and Learning research group (CALRG) and are 
reflected in this book, and then provides the context with a discussion of the inno- 
vative nature of the early Open University. The CALRG is then described briefly 
before the introduction of a further framework used in the book to analyse the fac- 
tors that make educational innovations successful, the Beyond Prototypes model. 
The last section describes the organisation and contents of the rest of the book. 


Introduction 


This book, “Educational Visions”, informs future developments in educational 
technology, by reviewing the history of computers and education, covering 
themes including learning analytics and design, inquiry learning, accessibil- 
ity and learning at scale. The lessons from these developments, which evolve, 
recur and adapt over time give an indication of the future in the field. The book 
informs readers about what is already known and demonstrates how they can 
use this work themselves. 

“Educational Visions” is based on the research of the Computers and Learn- 
ing research group (CALRG). Based at The Open Unversity, CALRG is the 
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longest-running UK research group in the field of educational technologyy. 
The Open University celebrated its 50th birthday in 2019, whilst the research 
group recently celebrated its 40" anniversary. The production of the book drew 
on the CALRG’s 40°-birthday celebrations and conference to extract themes 
and generate content of interest to a broad audience. 

The core principles of the approach to educational technology reflected in 
this book are: 


1 Teams can successfully teach any number of students at a distance 
2 Learning is accessible for everyone 

3 Teaching is adapted to meet learners’ needs 

4 Learners engage enthusiastically with STEM learning. 


These four principles give the research a practical emphasis linked to the 
application of educational technology to the benefit of learners. The prin- 
ciples also form the visions that are discussed and considered in the book. 
“Educational Visions” traces 40 years of research in this area, showing how 
these four visions are being achieved, identifying challenges that have been 
overcome; those that still remain, and extracting general themes in educa- 
tional technology. 

The structure of the book aligns with these four visions. In each case, the lead 
author on one chapter is a researcher with a global reputation, who examines 
the foundations of work towards this vision and how that work has developed 
over 40 years. A second chapter related to the vision is authored by research- 
ers currently active in this field, describing current work and future directions. 
This approach allows for a balance between historical analysis and current the- 
matic application. 

The structure of each chapter is also related to a framework (Beyond Proto- 
types) which explains why educational technology initiatives worldwide suc- 
ceed and why they often fail. This framework highlights that initiatives only 
succeed if they are guided by a vision and if they take account of the whole 
complex system of interacting factors that impact on educational technology 
(pedagogy, technical aspects, ecology of practices, technical context, student 
community, teacher community, technical communities, pedagogic research 
community, revenue generation, environment, policy context and funding). A 
fuller description of this framework and a graphical representation are pro- 
vided further on in this introductory chapter and chapter authors use this 
framework as a recurring theme. The aim is not only to introduce successful 
educational technology projects but also to make it clear why they are success- 
ful when so many others are not. 

Each chapter includes examples and/or case studies of significant work, the 
effect this work has had and its implications for the future. These examples 
include FutureLearn (a MOOC platform with more than ten million learners), 
the Conversational Framework (widely used to structure and understand online 
teaching and learning), iSpot (used internationally by citizen scientists) and 
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learning analytics tools. The remainder of this chapter briefly introduces The 
Open University, its mission, the CAL research group and the Beyond Proto- 
types Framework before offering some comments on the chapters that follow. 


The Open University 


The Open University was established in the UK in 1969 - and celebrated its 
50" anniversary in 2019. Its mission was and still is to be open to people, 
places, methods and ideas. During the 1960s, only a small percentage of the 
UK population attended university. In 1970, a year after the OU started, but 
before courses were available, the figure was 8.4%. A primary aim for The 
Open University was to extend access to higher education to a much wider 
population. While he was leader of the British Labour Party, Harold Wilson 
wrote an outline for the ‘university of the air’ and the Labour Party’s 1966 
manifesto included a commitment to establish such a university - building on 
the idea that the then relatively new technologies: radio, film and television, 
could bring university education within reach of a much larger audience. Once 
he was Prime Minister, Wilson appointed Jennie Lee as Arts Minister and to 
take charge of the project. 

The Open University (OU) Charter was signed in April 1969 and by 1972 the 
university had 36,000 students, increasing to 72,000 by 1979 when 130 under- 
graduate courses were offered. The OU’s core vision and mission, to be open to 
people and places, methods and ideas, has not changed significantly during its 
50 years. 


Open to people and places 


OU courses originally consisted of written course materials, sent out by post to 
students who studied part time. In addition, through a partnership set up with the 
OU, BBC (British Broadcasting Corporation) programmes were also made and 
broadcast in non-peak broadcasting times. Today, the BBC no longer produces 
broadcast programmes for OU courses, but the two organisations still work in 
partnership, co-producing programmes for all BBC ty, radio, digital and online 
channels/platforms. Printed course materials still play a role, but much is online. 

Students were allocated to regional centres and to tutors who worked in the 
geographical areas associated with these centres. The tutors supported the stu- 
dents’ work, marked their assignments and held tutorials at local centres during 
the evenings or on Saturdays. No qualifications were needed in order to start a 
course with The Open University: a radical idea at the time. 

Generally, students were able to study wherever they lived in the UK and even 
when working abroad. Since then there has been a growth in students working 
from non-UK locations although it is not always possible to take courses from 
anywhere in the world. 
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Open to methods 


The university pioneered new approaches to teaching, especially in STEM 
(Science, Technology, Engineering and Maths) areas where students did not 
have a traditional laboratory to hand. The idea of the Home Experiment Kit 
was developed. This involved posting out home chemistry sets, fruit-flies and, 
more recently, home computers. The very first science Home Experiment 
Kit included a small, compact microscope. Further examples are described 
in Chapter 8. 

Residential schools run at traditional university campuses were set up where 
students could use laboratories, have face-to-face lectures, and work collabo- 
ratively on projects. However, the first Chancellor of the OU also foresaw the 
importance of computers: 


I predict that before long actual broadcasting will form only a small part of 
the University’s output. The world is caught in a communications revolu- 
tion, the effects of which will go beyond those of the Industrial Revolution 
of two centuries ago. .... As the steam engine was to the first revolution, so 
the computer is to the second.” (Crowther, 1969) 


Extract from Speech by Lord Crowther, first Chancellor of The Open 
University at the presentation of the Charter, 23rd July 1969. 


Open to ideas 


Probably the earliest ‘big idea was that of ‘course teams’ that would develop 
and produce courses for the students. In addition to content specialists, these 
included editors, administrators and educational technologists. Course mate- 
rials underwent several drafts, which were commented on in course team 
meetings. Another significant example of the OU being open to ideas was its 
decision to develop a partnership with the BBC in 1971. This early collabo- 
ration meant that the programmes used in OU courses were made by BBC 
producers, in conjunction with academics, and so were able to benefit from 
the high production values of the BBC as well as including ingenious ways of 
helping students to visualise complex phenomena. 

Modern technologies were used from very early days as the University devel- 
oped ways of teaching science at a distance. A media mix that was heavily text 
based also included broadcast TV, audio and, from the mid 1970s, the use of 
computers. From the very beginning, the need for advising the University on 
the role of educational technology was acknowledged. A paper by the first 
Vice Chancellor, Lord Walter Perry, to the Planning Committee in July 1969 
emphasised that Open University staff should include those with skills in “all 
the modern methods of educational technology” (Perry, 1969). By March 1970, 
the Institute of Educational Technology (IET) had been established. One of the 
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IET’s roles was to “provide the Course Teams with continuous diagnostic feed- 
back as a basis for remedial guidance, revision and recycling”. 


The CAL research group 


The Computers and Learning research group (CALRG) was set up in 1979: 
housed in IET with founding members located in IET, Psychology and Com- 
puting. The first objective of the CAL research group was “to carry out a co- 
ordinated programme of research into ways computers can be used to improve 
the quality of education” (Jones and Scanlon, 1981). In their later edited book, 
Jones, Scanlon and O’Shea, (1987) explained that all the work described within 
the book “had a distance education (or training) setting in mind” (Preface, p.x), 
hence there was not only a strong commitment to researching teaching, but 
also a desire to understand how technology could improve teaching and learn- 
ing within the distance learning context of the OU. 

From the beginning, a significant part of the CAL research group’s work was 
carried out by PhD students: both full time and part time. Members of the 
group were drawn from across the OU and were involved in the University’s 
teaching in various ways: as authors, critical commentators, and evaluators, 
who shared “a strong commitment to improving education in general and dis- 
tance education in particular by the application of new information technolo- 
gies” (Jones, Scanlon and O’Shea, 1987, p.1). Thus, the part-time students chose 
areas of investigation that could be applied in their professional lives and to 
issues related to teaching. 


Models of Learning: a cognitive science approach 
to understanding the learner 


During the late 1970s and the 1980s, the CALRG was influenced by the rise 
of cognitive science, which embraces a number of different disciplines: (see, 
e.g. https://plato.stanford.edu/entries/cognitive-science/), and, in particu- 
lar, the application of artificial intelligence (AI) research. Such research was 
often located in the United States (e.g. Carnegie Mellon, MIT, Stanford) and 
in the UK in Edinburgh, Sussex, and the MRC Applied Psychology Unit in 
Cambridge. One founding member of the CALRG developed a “self-improv- 
ing” quadratic tutor, (O’Shea, 1979), and another researcher (Eisenstadt, 1982) 
wrote a unit on AI for a cognitive psychology course. 

Following Al-influenced approaches, some CALRG researchers were inter- 
ested in developing production systems - computer programs consisting of 
a set of rules; ways of deciding different rules, and an interpreter to run the 
system - for psychological modelling, building theories about how people 
learn. To do this they collected student protocol data - where protocols are 
concerned with the process of carrying out a cognitive task (Ericsson and 
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Simon (1980). This data could be used to construct production rule models 
which could then be used to predict the kinds of error that students were likely 
to make. Such models are valuable for examining learning but can also be 
embedded within tutoring systems (see for example O’Shea’s “self improving” 
tutor, O’Shea, op. cit.). 

Perhaps predictably, the fields and specific areas where such work was 
applied were those with sufficiently well-defined problems to benefit from 
such an approach. These included maths, physics problem solving, program- 
ming and graph interpretation skills. Many of the new approaches to teach- 
ing pioneered by The Open University related to STEM subjects and so there 
was a particular need to conduct research into teaching in these areas. This 
emphasis on STEM areas has persisted, and is reflected in Vision 4: Learn- 
ers engage enthusiastically with STEM learning, although there has also been 
research in many non-STEM areas, including language learning, which is 
discussed briefly below. Overall, there has always been a strong interest in 
supporting student learning through designing instruction to help improve 
student performance, so the relationship with teaching is strong. Early goals 
also included theory development, e.g. understanding how physics problem- 
solving skills were developed. 

Some developed models of learning that were used within courses. For 
example, O’Shea developed a computer game, modelled on the fraction buggy 
(Brown and Burton, 1978), which was embedded into teaching on an influ- 
ential educational course for teachers: “Developing Mathematical Thinking” 
Indeed, cognitive science research had a significant impact on mathematics 
education more generally, see Siegler (2003). 

CALRG research and evaluation have also fed into course design, some- 
times directly and at other times more indirectly through studying students 
closely and developing an understanding of how the design of course mate- 
rials influences students’ understanding and learning. Good learning design 
is particularly important where students are learning at a distance. The pro- 
duction of Open University courses included a cycle of draft material which 
was commented on by the course team, which typically included both design- 
ers and media experts. In addition to the internal comments and suggestions, 
external readers would also read and comment on the materials. Some courses 
were developmentally tested - involving a more complete ‘run-through; often 
by students. 

As noted by Jones, Kirkup and Kirkwood (1992): “... it is particularly impor- 
tant to test out any element of practical work, whether it is a laboratory-style 
home experiment or the use of a computer, as students will be on their own if 
and when any problems occur” (p105). Their work on the Microcomputers for 
Schools project in the 1980s described how materials developed for teaching 
teachers were designed. They drew attention to the practical work texts, which 
were laid out in a three-column format, “where the first column gave the keypress 
required, the second gave a photograph of the screen display that appeared as a 
result of that action, and the third column provided a commentary” (p106). 
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The Beyond Prototypes Framework 


So far we have discussed how some of the educational technology research at 
the OU, conducted by members of CALRG, has fed back into and influenced 
the design of OU teaching material, its courses. However, there is also the 
larger question of how educational technology research, sometimes referred 
to as technology-enhanced learning (TEL) can be applied in education more 
generally and, when the research is focusing on development, how it can lead 
to useful and usable products. 

A recognition that successful TEL innovations needed to be interdisciplinary 
led to the establishment of the ESRC (Economic and Social Research Council) 
TEL programme, which ran from 2008 to 2013. This programme awarded funding 
to eight TEL projects. Reflecting on these, Richard Noss, who directed the pro- 
gramme, wrote: “So the issue is this one: in general, despite the fact that all projects 
successfully designed and built effective prototypes of systems: the question is how to 
move from prototype to product? (Scanlon et al., 2013, p. 4). He continued: “So this 
report is concerned with how to move from academic research and innovative proto- 
types to effective and sustainable products and practices.” (Scanlon et al, op. cit. p.5). 

In order to do this, an interdisciplinary team of educational technologists 
analysed and evaluated TEL research and selected key examples to study in 
further detail. Drawing on these they developed a framework: “Beyond Pro- 
totypes’, which is used to frame and consider both the historical and the con- 
temporary research discussed in the book. The framework is based on four Key 
Insights from the research (see Scanlon et al, op. cit. pp. 5-8). 


1 TEL involves a complex system of technologies and practices. In order 
to embed significant TEL innovation successfully, it is necessary to look 
beyond product development and pay close attention to the entire process 
of implementation. 

2 Significant innovations are developed and embedded over periods of years 
rather than months. Sustainable change is not a simple matter of product 
development, testing and roll-out. Persistent intent is needed. 

3 TEL innovation is a process of bricolage. This process includes informed 
and directed exploration of the technologies and practices required to 
achieve an educational goal. It involves experimentation to generate fresh 
insights, and creative use of available resources. It also requires engage- 
ment with a range of communities and practices. 

4 Successful implementation of TEL innovation requires evidence that the 
projected educational goal has been achieved. Reliable evaluations must be 
carried out; their findings disseminated and acted on. Methods of evaluation 
are required that can be applied to processes of innovation and to institutional 
change, as well as those that can be applied to shifts in technology usage. 


At the heart of the Beyond Prototypes model is Key Insight 1 - the “TEL com- 
plex. The framework is represented in figure 1 which is reproduced below: 
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Figure 1.1: The Beyond Prototypes Model of the TEL Complex. Adapted from 
Figure 1 in Scanlon et. al. (2013) with thanks to the TLRP/TEL programme 


for permission to use this figure. 


In order for TEL innovation to be successful, key elements must be consid- 
ered. These are: 


e pedagogy: “an extremely complex and distinctive process which involves 
both student and teacher engagement, delivering a set of educational ser- 
vices by means of specific channels.” (Scanlon et. al., 2013 p.28); 

e technical components that support the pedagogy in order to achieve the 
aim of enhancing learning in a specific way; 

ethe ecology of practices and technical context. These are important 
because any TEL will be implemented in a specific context. For example, 
internet coverage may be sporadic or unavailable (see, e.g. Gaved and 
Peasgood, 2017); 

e the communities involved in the TEL complex, including students, teach- 
ers, researchers and technical developers or supporters. 


Whilst it would be beyond the scope of this book to apply the complete frame- 
work for the research discussed, each chapter illustrates how aspects of the TEL 
complex work in practice. So, in Chapter 7 Rienties and Jones describe how 
the learning analytics research currently being undertaken involves a number 
of different communities in the University and how ‘data wrangling’ (a pro- 
cess which provides learning analytics information about University courses) 
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involves a series of discussions with the academics involved in developing 
those courses. 

Continuing work with the nQuire tool illustrates how persistent intent 
works (Key Insight 2). The nQuire tool was originally developed during a 
three-year project funded by the ESRC/TEL programme (2007-2010); Shar- 
ples et al (2015). 

At the end of the successful project, a prototype tool was available. This 
required considerable further development, and was taken forward in a num- 
ber of projects, (see e.g. Sharples, Aristeidou, Herodotou, Mcleod and Scanlon, 
2019) as described by Scanlon and Herodotou in Chapter 9. In 2019 the nQuire 
tool was used by over 200,000 people as part of a citizen science project, Gar- 
denwatch, to fill in a gap in information about the nature of UK gardens as 
habitats for wildlife. 

Much of the emphasis of CALRG research has been on STEM subjects. 
However, the group has also been involved in research outside STEM and it 
continues this trajectory. Notable examples of research into how educational 
technology can support learning in other areas and contexts includes, for 
example, the use of mobile technologies to support language learning especially 
in informal contexts. This research has also been applied to support migrants’ 
language learning, see, e.g. Kukulska-Hulme (2019) as noted in chapter 10. 
Another example is the work carried out by Charitonos on how microblog- 
ging (particularly the use of Twitter) can help to support and connect learning 
across museums and classrooms (see, for example Charitonos, 2019). A final 
example is Jones’s work on affective issues in learning technologies, including 
using computer-mediated role play for investigating and supporting children’s 
socio-emotive development and expression (Jones, 2010). 

Even when research has originated in STEM contexts, it is often applied 
and further developed in numerous other non-STEM contexts, or in contexts 
where the types of scientific inquiries are very varied. One good example of 
this is the nQuire project, discussed by Sharples in the concluding chapter. The 
first nQuire project developed and evaluated an inquiry tool that could be used 
on mobile devices or on a computer to support school children’s personalised 
scientific inquiries. Such inquiries included investigating whether an Urban 
Heat Island existed in the new city where one school was located. Other inquir- 
ies investigated healthy eating, bird feeding behaviour and how cheese rots. 
Parts of the investigations took part in the field, other parts at home and in 
the classroom. 

This line of research is a good example of persistent intent. It has continued 
over a very long period, developing a platform that can support inquiries at a 
large scale, with a very broad definition of science. 

The work on learning design, and our understanding of the learning process, 
which began in the 1970s (see chapter 6) is also applied to a very wide range of 
topics. In chapter 7, Rienties discusses its application to OU course or module 
design, whilst Ferguson (chapter 3) and Sharples (chapter 10) focus on different 
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aspects of how research into the pedagogic process and learning design has fed 
into the development of MOOC platforms. 

The remainder of this book is organised as follows: (this chapter is included 
in the list for completeness). 


Chapter 1: Introduction 

Ann Jones, Eileen Scanlon and Rebecca Ferguson 

Introducing CALRG and giving the background to the group; setting out the 
visions, and placing them in context; introducing the framework and its impor- 
tance for educational technology. 


Chapter 2: Teaching and Learning at Scale: Foundations 

Patrick McAndrew 

Studies of how distance learning can be supported by TEL and how these meth- 
ods can scale. This chapter relates to Vision 1: Teams can successfully teach any 
number of students at a distance. 


Chapter 3: Teaching and Learning at Scale: Futures 

Rebecca Ferguson 

Recent developments in this area, particularly massive open online courses 
(MOOCSs.) This is the second chapter relating to Vision 1: Teams can success- 
fully teach any number of students at a distance. 


Chapter 4: Accessible Inclusive Learning: Foundations 

Tim Coughlan, Kate Lister, Jane Seale, Eileen Scanlon and Martin Weller 

The development of open learning, and work on accessibility. This chapter 
relates to Vision 2: Learning is accessible for everyone. 


Chapter 5: Accessible Inclusive Learning: Futures 

Tim Coughlan, Kate Lister, Jane Seale, Eileen Scanlon and Martin Weller 
Current work on open learning and accessibility and future directions. This 
is the second chapter relating to Vision 2: Learning is accessible for everyone. 


Chapter 6: Evidence-Based Learning: Foundations 

Ann Jones, Bart Rienties and Canan Blake 

Developing a coordinated programme of research into ways computers can 
be used to improve the quality of education. This chapter relates to Vision 3: 
Teaching is adapted to meet learners’ needs. 


Chapter 7: Evidence-Based Learning: Futures 
Bart Rienties and Ann Jones 


Introduction II 


How this research programme has developed, with a focus on the relationship 
of learning design and learning analytics. This is the second chapter relating to 
Vision 3: Teaching is adapted to meet learners’ needs. 


Chapter 8: STEM Learning: Foundations 

Eileen Scanlon, Christothea Herodotou, Denise Whitelock and Chris Edwards 
Origins of work on learning and teaching STEM subjects at a distance. This 
chapter relates to Vision 4: Learners engage enthusiastically with STEM 
learning. 


Chapter 9: STEM Learning: Futures 

Christothea Herodotou, Eileen Scanlon and Denise Whitelock 

Current and future work on areas including citizen science, inquiry learning 
and virtual field trips. This is the second chapter relating to Vision 4: Learners 
engage enthusiastically with STEM learning. 


Chapter 10: Visions for the Future of Educational Technology 
Mike Sharples 
Drawing the themes together. 
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CHAPTER 2 


Teaching and Learning at 
Scale: Foundations 


Patrick McAndrew 


The Open University (OU) offered a radical change in how higher education was 
considered in the UK from the 1960s. A fundamental change was to allow access 
to those who wanted to learn, without their needing to show prior success in quali- 
fications. Alongside this expansion in offering access there also needed to be a 
rethink in how learning could operate at scale. Higher education has a tradition of 
individual responsibility that was not suited to the complexity of the task, rather 
the OU needed to take a team approach to designing, building and operating the 
learning experience. 

Addressing such complexity is one part of the model in Beyond Prototypes 
(introduced briefly in Chapter 1), recognising the complexity that occurs when 
applying Technology Enhanced Learning (TEL). Each part of the model can be 
seen to act in the development of the OU: persistence in achieving the aims of the 
OU to meet the needs of a new body of students; bricolage in being prepared to 
experiment with new ideas and new technologies; and, an underlying depend- 
ence on evidence to help understand and address the problems that are faced 
at scale. 
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Beyond Prototypes 


The Beyond Prototypes model sets out four different aspects of innovations 
that are summarised in the report (Scanlon et. al., 2013): 


1. The TEL complex 

“TEL involves a complex system of technologies and practices. In order 
to embed significant TEL innovation successfully, it is necessary to look 
beyond product development and pay close attention to the entire process 
of implementation.” 

2. Persistent intent 

“Significant innovations are developed and embedded over periods of years 
rather than months. Sustainable change is not a simple matter of product 
development, testing and roll-out.” 

3. Bricolage 

“TEL innovation is a process of bricolage. This process includes informed and 
directed exploration of the technologies and practices required to achieve an 
educational goal. It involves experimentation to generate fresh insights, and 
a creative use of available resources. It also requires engagement with a range 
of communities and practices.” 

4. Evidence 

“Successful implementation of TEL innovation requires evidence that the 
projected educational goal has been achieved. Reliable evaluations must be 
carried out; their findings must be disseminated and acted on. Methods of 
evaluation are required that can be applied to processes of innovation and to 
institutional change, as well as those that can be applied to shifts in technol- 
ogy usage.’ (Scanlon et al, 2013, p6-7] 


Considering the OU through the Beyond 
Prototypes framework 


In this chapter those four aspects of the Beyond Prototypes model emerge from 
the work of the OU, including the Computers and Learning research group 
(CALRG) as part of the Institute of Educational Technology (IET). 

The challenge of forming the first modern open university was undeniably 
complex, and the solutions proposed recognised both the complexity in the 
situation and also space for further development. The first section of this chap- 
ter considers how complexity was addressed by those working to bring the 
university together as they thought through how teamwork, and the role that 
educational technology needs to play, could work in practice. 

In the second section the persistence of the work is considered. In particu- 
lar considering how the university addressed the challenge of being open to 
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learners. One of the fundamentals of the approach to distance education taken 
by The Open University is its open entry system. Open entry allows anyone 
to choose to study any subject that is offered. There are no prerequisites; at 
one stage this also meant that students could choose to study at different levels 
(though the fee system now in place makes that choice much more limited). 
This does not mean though that all choices will lead to equal success or that 
success can be measured on the same basis for all learners. Over time systemic 
threats to operation have occurred driven from funding models that have 
impacted unfairly on the part-time sector. These pressures have meant the OU 
has needed to show persistence in remaining true to its mission and yet avoid 
the complacency inherent in being a market-leader. 

The third element is experimentation; “bricolage”. The OU arose through a 
process of experimentation and willingness to try alternative approaches to 
support students to learn at a distance. The Open University came into oper- 
ation at a time when new tools and new ways of thinking were available to 
reconsider the way correspondence education worked, and so provide a fuller 
experience of distance education that would be accepted as comparable to that 
from more conventional higher education providers. Being prepared to try new 
approaches has been very successful, however there are also risks in then set- 
tling into an accepted approach and providing a legacy position. The difficulty 
of being an innovator from a dominant position is recognised (Rogers, 1995) 
and can lead to susceptibility to losing position to other more disruptive inno- 
vators. At several points in the OU’s history such disruption has threatened, 
as technology offered alternative ways for other providers to operate without 
the investment at scale of the OU. Nonetheless, the OU remains the leading 
distance and part-time provider of higher education in the UK. This is in part 
due to a willingness to itself disrupt its own models for working. 

The fourth component of the Beyond Prototypes model is evidence. The OU 
established an evidence-based approach from the beginning, driven by the 
need to prove the quality of its approach and show value. Through research 
mainly carried out in the Institute of Educational Technology, the OU has 
always surveyed its students at scale and monitored the performance and satis- 
faction of its students. The role of evidence has been refreshed in more recent 
years through the availability and use of analytic measures. 


Meeting complexity through teams at the OU 


The establishment of The Open University arose out of a mix of political and 
practical ambitions in the 1960s that made its existence at first uncertain for 
reasons that included “demand was not proven, many students would leave 
early and degree-level work could not be taught in such a way” (Weinbren, 
2015, p11). On the other hand, there had been considerable thought and plan- 
ning into the idea of the OU, with the ideas captured in the original Planning 
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Report (HMSO, 1969). That report set out the different components of 
OU teaching and also how the OU might be structured in order to address 
those elements. 

A key starting point was that the aim was not just to produce a correspond- 
ence approach to higher education. Instead the full range of components that 
made up a university were considered. That meant as a consequence that a 
more complex framework was required in designing The Open University 
than had been applied in established correspondence education. In keeping 
with the original concept of a “University of the Air” the Planning Report 
highlighted a role for broadcast as a replacement for the lecture of more con- 
ventional higher education. The report noted the “logistical advantages” of 
the lecture with broadcast providing scope to improve on quality and econo- 
mies of scale so that they were “likely to achieve results at least as good as 
and often better than those secured by the normal live lecture”. Six educa- 
tional aims were assigned to the use of broadcast. Three being of benefit to a 
general audience: 


a) Allow presentation of topics with high impact. 
b) Share cultural value to a wider audience. 
c) Promote awareness of the possibilities of higher education. 
Together with three being of benefit to those who became students: 
d) Support preparation so that there would be lower “fall-out” rate of those 
who do then sign-up. 
e) Avoid the need for students to travel to attend events. 
f) Help students to feel part of the university and in contact with its staff. 


The use of broadcast was only seen as one element in the OU approach to 
teaching, albeit one that attracted attention in establishing the OU. Those who 
founded the OU were clear that it was “neither practically possible nor peda- 
gogically sound to rely on broadcasting as the principal or exclusive means of 
instruction in an operation designed to provide disciplined courses of uni- 
versity level” (HMSO, 1969, p6). Instead it was proposed that a staff body be 
established that could support the mix of broadcast with printed materials 
and also offer the students the support that was needed. Some guidance for 
the aims of the OU came from work in the US, and several early appointments 
and visiting consultancies drew on experiences in independent and corre- 
spondence study there. While it is hard to give single credit for such work it is 
worth noting one contribution as described by Moore (1990) of the previous 
experience by Charles Wedemeyer in attempting to establish an Articulated 
Instructional Media project (AIM) at the University of Wisconsin to offer 
degree level education at a distance. Quoting Wedemeyer, Moore states that 
AIM “was an experimental prototype with three fatal flaws: it had no control 
over its faculty, and hence its curriculum; it lacked control over its funds; and 
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it had no control over academic rewards (credits, degrees) for its students. 
The implications were clear: a large-scale, non-experimental institution of the 
AIM type would have to start with complete autonomy and control” [quoted 
in Moore, 1990, p. 292.]. Whether drawn directly on this experience or not, 
these three points do appear to have been considered in the formation of the 
OU as an autonomous entity structured to have academic faculty, independ- 
ent finance, and awarding powers. 

The structure was also planned in greater detail with the Planning Report 
proposing six different substructures, four areas that were considered essential, 
with two further areas seen as desirable. 

The four core areas can be summarised as: 

1. Central administrative office - to take control of the finances and to 

recruit and administer the students. 

2. Academic departments - faculty on academic contracts that would be 

responsible for the content as a whole. 

3. Specialised academic related staff to support the methods of teaching and 

media required, including broadcast through the BBC. 

4. Administration of the part-time tutors and student counselling services, 

this was seen as being distributed across regions and nations. 
The further two areas outlined in the Report were: 

5. An operational research unit to be “established as an early priority” to 

undertake studies in support of new methods of teaching. 

6. To set up its own publications department. 


The division of labour in the structure that was planned was very much reflected 
in the eventual setup of the university, with integration of each area around the 
structuring of a “course team” approach to ensure that responsibility for the 
student experience, and the materials and media that supported it, would be a 
shared one. The ambitious aim for the OU was to establish four foundational 
courses for study across key disciplines, to support those with different media, 
including broadcast, and to offer those courses to 25,000 learners within two 
years of being established. The “course team” emerged as a concept that has 
since become a core element of the university's success and provided resilience 
to several challenges. That the concept was not fully formed was acknowledged 
by those who worked on it at the time as “an enormous bit of improvisation” 
and the contributions needed had to rapidly evolve, “nobody for example real- 
ised that you actually had to have professional editors” (Mike Pentz, Dean of 
Science, speaking in 1979) (Open University, 1979). The lessons of that early 
stage were that not only did content have to be produced, it also needed to be 
integrated into a learning experience and the students supported to achieve 
the right outcomes. The role of research and evidence was important and from 
an early stage data on student performance was included, through surveys of 
students at scale and targeted projects to understand student needs. 
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Formation of IET, the Institute of Educational Technology 


In forming The Open University as a complex organisation there was recogni- 
tion that there was a need to invest not only in the discipline-based materials 
needed by the students, but also in the way that learning would operate for the 
students. The Planning Report had noted the need for research into educational 
technology: “We propose, therefore, that an operational research unit of the 
University be established as an early priority in order to undertake the neces- 
sary studies. Indeed the continuation, as an integral feature of the University, of 
experimental work particularly in relation to the learning process may eventu- 
ally prove to be one of the University’s distinctive contributions to education 
generally” (HMSO, 1969, p16). (Experimentation certainly did continue; the 
current and next planned phase for such work is discussed in the next chapter.) 

The OU was established in 1969, gaining its charter in April 1969, and dur- 
ing its first year a decision was made to establish a unit to bring together staff 
development and research. A paper provided to the Council of the University 
(Perry, 1969) stated that the “role of The Open University as a major inno- 
vator in education” required “not only academic personnel distinguished in 
their respective disciplines, but also staff with special skills in all the mod- 
ern methods of educational technology.” The paper therefore “recommended 
the immediate establishment of an Applied Educational Sciences Unit” with 
responsibilities that included “service for course teams and contributing 
towards a professional development programme for University staff” Research 
was flagged as likely to increase as “academic staff become more conversant 
with the techniques of educational technology, and the workload with course 
teams decreases”. These actions are also reflected in a later book considering 
the early days of the OU authored by the first Vice Chancellor, Walter Perry, 
saying “The Planning Committee had recommended most strongly an on- 
going programme of research into the operation of the University. ... I there- 
fore proposed that the Department of Applied Educational Sciences should be 
combined with the group of Research Officers into a single Institute. ... My 
suggestion was accepted ... and the Institute of Educational Technology was 
born. (Perry, 1977, p81). The role of the Institute of Educational Technology 
was summarised in a further paper to Senate in 1970 in terms of the impact 
needed on OU instructional materials: 


“(1) They will all have been extensively tested and validated on representa- 

tive samples of students and volunteers. 

(2) They will make provision for individual differences, by permitting some 
choice of route and rate towards the course objectives. 

(3) They will utilise the various media and supporting services to best 
advantage. 

(4) They will demand participation from the student and will provide him 
with frequent assessments of his progress. 
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(5) They will provide the Course Teams with continuous diagnostic feed- 
back as a basis for remedial guidance, revision and recycling” (Open 
University, 1970, p1). 


The formation of IET provided one indication that the OU considered that the 
way in which it taught needed to both be different to conventional approaches 
and to evolve and change over time. The aim was to be innovative and to 
embrace the use of technology where it was appropriate to do so. 


Persistent approach in developing distance education 


The Open University has operated for 50 years on the basis that students can be 
enabled to learn at a distance. The very existence of the university is a testament 
to its persistence in ensuring the approach works. Having started by recruiting to 
its then maximum capacity of 25,000 students in 1971, it is the UK’s largest uni- 
versity in terms of numbers of students taught, both expressed as a total number 
and as full time equivalent. Over its 50 years more than 2 million people have 
become its students, and nearly 500,000 have received qualifications. In 2018 
it had over 170,000 students studying (though that is below the peak of over 
250,000 in 2011/12). In addition, it offers free learning to over 7 million learners 
through its open educational resources (described further in forthcoming chap- 
ters) and a further 2 million supported by FutureLearn (Chapter 6). This is all in 
the context of a university that was created in uncertainty and amid questions as 
to whether it was offering education to an appropriate group of people. 


Fundamentals for learning at scale 


From its start The Open University adopted an open approach to higher edu- 
cation that may still be seen as going against accepted practice that access to 
such education is to be earned through achievement in earlier assessed educa- 
tion. The university system that developed in the western world and dominates 
in the world today is based on a model of scarcity. That model sees access to 
recognised education as less available as the levels increase. In particular post- 
secondary education is positioned as being for a privileged few. In relatively 
recent times that was reflected in a very clear way in the design and operation of 
the education system in the UK, with growth in the last 50 years to give access 
for more of the population into tertiary education. In 1960 approximate 5% of 
the population would attend university between the ages of 17-30, by 1970 this 
had risen to nearly 15% where it remained throughout the 70s and 80s. Then 
in the 1990s there was new policy and ambition with a push to attain 50% that 
led to increases in the numbers entering tertiary education, with 49% being 
claimed in 2017 (Guardian, 2017). 
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Figure 2.1: Age Participation Index of 17-30 year olds in university (%) 
1960-2001: Adapted from Chowdry et al (2010). 
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Figure 2.2: Higher Education Initial Participation Rate for England. Data from 
the Higher Education Statistics Agency (HESA) 2018. 
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The earlier trend towards greater entry to university in England is shown in 
Figure 2.1 above (adapted from Chowdry et al. (2010), using data from Fine- 
gold (2006)) and more recent data from HESA(2108) in Figure 2.2. Even with 
increasing attendance there was a need to diversify provision (Finegold, 2006). 
With 50% provision there remain both “the other half” and those that had left 
education before it expanded. Yet the dominant model remains one in which 
university study is undertaken during a period away from home and within 
a separate process from that of ‘work. Privilege remains built into the system 
with access gained on prior attainment, by seeking to filter out those who are 
not suitable at the start of the process. Prior qualifications are required with a 
requirement for “2 A-levels” an underlying assumption for eligibility for study. 

The OU represents a persistent approach to push back at this existing model, 
through understanding both some of the truth in assumptions, and in looking 
at how to address them. A clear example of persistence is support for entry 
into study. An open entry approach to study inevitably means that on some 
measures The Open University will not look as successful as other universities. 
Past success is a good indicator of future success and at the point of entry that 
is most efficiently judged through grades achieved in previous formal examina- 
tions, with a correlation between those and future results identifiable, though 
weak (Birch and Rienties, 2014). While accepting such a link provides an over- 
all picture, it is not the case at an individual level that learners with low or no 
previous qualifications will not succeed. Indeed the idea that there is a “lim- 
ited pool of ability” has been rejected in previous reviews of higher education 
(Dearing, 1997). 


Do qualifications matter? 


Being open entry and operating at scale means that it is possible to see the 
different performance of those from different backgrounds. Four lessons have 
been learned here: 


1. Anyone may succeed. So using data such as prior study and economic 
background to determine whether someone should be allowed to try 
to study means that some of those who would have succeeded will have 
been excluded. 

2. Prior qualifications are an indicator for initial success. Perhaps not sur- 
prisingly those who have already shown success in earlier study are more 
likely to succeed, however for all abilities of student there are similar 
points where they struggle and determination is needed. 

3. Once success in OU study has been achieved then previous study his- 
tory is almost irrelevant. This means that it is vital to support students to 
achieve that initial success and to set them on the path to success. 

4. The measure of success is not necessarily the same for the student and the 
institute. The focus institutionally and culturally is often on a qualification. 
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This is reflected back if students are asked to state their aims. Overwhelm- 
ingly (nearly 90% of) OU students will indicate that their aim is to achieve 
a degree. However, in practice students may gain advancement or satisfac- 
tion without meeting the degree. In the past this was built into the highly 
modular study structure of the OU. Current structures risk labelling those 
who leave prior to the degree as failures at system level (so that in league 
tables or funder progress models these are not successes), and at insti- 
tutional level (with metrics set on final outcomes rather than progress). 
Unless success is identified at the student level this risks individuals seeing 
themselves as failing, despite possibly having achieved academically more 
than they had ever done before. 


Modelling performance 


One revealing indicator for the way in which different factors impact on the 
success of students is the analysis of the cohorts of students used to help judge 
whether a module at The Open University is performing as expected. Such 
analysis allows comparison with historic measures of performance for student 
completion and attainment, adjusted to allow for the impacted of the student 
body that is recruited. While demographics for an individual are not a sufficient 
reason to infer their success or failure, at scale such demographics can help us 
understand whether a group as a whole is likely to perform better or worse. An 
historic factor analysis has been used for the last 10 years to take into account 
all measures available. Some of the factors considered are shown in Table 1. 


Table 1: Factors in predicting student performance in Open University 
modules (information provided by Vicky Marsh and Jim Peard of The 
Open University). 


Student motivation Module factors Previous student 
factors (demographics) | (choice of study) experience 
Occupation Discipline area Previous OU study 
Gender Level Previous study success 
Age Credits from the module Previous educational 
Credit transfer Number of assessments in the | qualifications 

module 


Credits gained in same 
discipline area 

Credits gained at same level 
of study 


The details of the modelling are interesting, though not important in the con- 
text of this chapter. Here the importance is that the 10 years of data show a high 
level of consistency for the factors that emerge and in differences between initial 
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study and continued study. For initial study one of the top factors is highest 
previous educational qualification gained before study, for continuing students 
this does not appear as a top 10 factor in the analysis of statistically significant 
factors in determining student success. For new students three of the other top 
10 factors are also related to their background while for continuing students 
there is only one remaining demographic factor, relating to the market segment 
that they come from rather than their personal background. In simple terms 
this implies that from OU data it is only at the start of study that likely success 
can be judged from personal background or achievements in earlier education. 
Once study has got underway then previous successful study within the OU is 
the strongest predictor. This indicates the short-sightedness of building a system 
of higher education that first filters out people based on already achieved quali- 
fications, rather than giving them a chance to develop and prove themselves. 


Commencing study 


The difference in achievement in early study that does depend on previous 
preparation has also been addressed in the OU approach with initial study 
seen as part of learning to learn rather than as a way to filter out. The OU has 
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Figure 2.3: Available Badged Open Course: Succeed with Learning. 
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persistently looked to provide routes to support early study. Initially this was 
through gateway study with partner organisations, notably the National Exten- 
sion College, and through foundation stages in the OU degrees. More recently 
it has supported those returning to education through programmes of Open- 
ings courses and, currently, through a range of Access provisions. Some of those 
changes reflect development over time, others the response of the OU to exter- 
nal factors. The value of the Openings approach was also demonstrated when it 
was used to support students facing difficulty in their study in US Community 
Colleges. In the Bridge to Success project (Law et al., 2012), material that was 
part of two Openings courses was reworked into open educational resources to 
support Learning to Learn and Succeed with Math. These resources were then 
used flexibly across a range of contexts by US Community Colleges, universi- 
ties and other organisations such as charities. The success of these alternative 
bridging models both to inspire educators (Coughlan et al., 2013) and provide 
support for students (Pitt et al., 2013) showed that this is often an under-sup- 
ported area where approaches in place in the OU can transfer successfully into 
other contexts. Updated versions of both of these courses remain freely avail- 
able as Succeed with Learning and Succeed with Maths with badged recognition 
(Badged Open Courses) on the OpenLearn site (Law, 2016), Figure 2.3. 

The current range of Access modules were designed to fit with the need for 
students in some parts of the UK to be eligible for loans to pay for study. This 
meant that the modules needed to become larger and eligible for accreditation 
as pre-university study. The reworking also allowed other changes so that the 
modules contain additional online study and revised assessment (Hills et al., 
2018), both of which prepare the students more fully for further study (Butcher 
et al., 2018). While it might be tempting to see such Access modules as the poor 
relation in contrast to the main stream modules, in practice they are among the 
most highly rated modules for student satisfaction and have shown their value 
in improved performance, especially for those otherwise at risk, such as those 
with low previous educational qualifications. This shows action by the OU to 
give students, who would otherwise be in categories that might struggle with 
initial study, a route that can lead them to build up success with the OU and so 
improve their chances of further success. 


Bricolage: Experimental approach to the learning experience 


The OU supports learning at a distance for higher education in a changing 
environment both in terms of what students want and in how the technology 
can support it. By necessity the OU has needed to operate an experimen- 
tal approach; initially in taking new steps into how to provide the scale that 
is needed, then to retain an innovative approach that evolves to make use 
of technology for educators and for students. Major changes have included 
adoption of online virtual learning environments, use of online conferencing 


Teaching and Learning at Scale: Foundations 25 


and remote connections, and changes in how we assess students. These 
though have not come in isolation as one-off decisions, rather in each case 
there is a history of experimentation at a range of scales and of learning from 
external experiences. 


Materials, cascaded support and feedback 


The concept of materials as the basis for learning may initially feel wrong. That 
we can package up learning seems to draw us back to the Nurnberg funnel 
concept that we can simply provide the content and pour it in. The materials 
may have moved on from written texts to also include the use of other media 
such as broadcast, video and audio however if it is all there and can be sent 
in a big cardboard box then the student will be able to spend the time and 
come back having “learnt” it all. This, of course, is not the case and the idea of 
materials-based learning is not that the materials are everything, rather that 
they can provide an alternative to the high-contact approach and synchronous 
presentations seen in more traditional lectures. Furthermore the design of 
the experience around the materials can build in the tasks, activities, support 
and assessment that make learning more likely and do this in a way that can 
be scaled. 

The design process for OU courses contained two innovations in particular. 
The first innovation is that the teaching component should be available within 
the content: while content is not necessarily king it is a vital component. The 
second innovation is that content would be closely supported by feedback to 
the learner. 

Feedback has for some time been an unacknowledged element in education 
and learning. In Hattie’s analysis (Hattie, 2008) of effects of educational innova- 
tions, feedback is ranked as one of the areas having highest likely impact, and 
yet this is a factor that often is either ill-defined or side-lined. The National 
Student Survey includes a rating for feedback. Assessment and feedback is the 
area in which the OU has consistently ranked highly in comparison with other 
UK universities. In 2019, it came first out of all higher education institutions 
with 85% compared to an average of 73% (OfS, 2019). For feedback to be so 
strong in the case of an institution where feedback has to be given at a distance 
is perhaps at first surprising. However this has been achieved through integra- 
tion of feedback into the learning design and assessment of materials, to ensure 
feedback is provided at a scale that allows sufficient time and attention to be 
given to the individual. 

Feedback at the OU operates both through its role in the design of the learn- 
ing experience and in how support is provided. Materials are designed to be 
interactive and involve the learner in line with the materials-based approach. 
The materials need to talk to the learner, and ask them to be active. This can 
be achieved in print-based content by a combination of the language used and 
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the addition of accompanying motivating activities, such as through watch- 
ing broadcast programmes or through experimental kits. With availability of 
online content, interactivity is also through online activities, discussions and 
simulations, as part of the designed overall approach so that the structure of the 
module is made an active experience. 

The second aspect of how feedback operates at the OU is through the cas- 
caded model of support. An Associate Lecturer will typically support a group 
of approximately 20 people who are studying a module together. They will have 
started at a common point and while there will be flexibility in terms of day to 
day pacing of study there will be key synchronisation points where students 
need to supply material for assessment. Such assessment is designed to help 
the students gain understanding and receive feedback through the marking of 
the work. Typically the work will also be marked and form part of the assessed 
outcome of the module, however by design it is acting in a formative way - 
to help the student learn. The tension between marked work and feedback is 
well recognised in the literature with studies showing that feedback is the more 
important component for learning, while marks provide the higher motivation 
to complete, in line with other work where optional components for learning 
inevitably have a lower priority. 


Distance learning works 


Working in education or training it is very common to come across the atti- 
tude that there is nothing as good as face-to-face teaching and that anything 
that differs from getting people in the same place to learn is a compromise. 
This belief has been challenged over the years with correspondence education 
developing from the 18th century with the University of London, being the first 
higher education teaching establishment to provide correspondence education 
at scale. With the formation of The Open University in 1969, as described ear- 
lier, there was appreciation that the range of tools available could move on from 
correspondence models to alternative ways to connect with students. Overall it 
represented a rethinking of the approach to learning, considering the “Educa- 
tional Technology” of how people learn. 

The method that emerges from this work is one that values all the components 
in student activity. These include designing shared events, such as broadcasts, 
that motivate student engagement; providing content direct to the student, 
as written texts and other media (including physical experimental kits) that 
incorporated all that was needed to cover the required syllabus; and building in 
support through contact with tutors either face-to-face or through technology, 
such as via telephone. Bringing the whole method together was an approach to 
assessment that built in staged feedback from tutors that is tailored to the indi- 
vidual student and advises them how to improve, whilst ensuring a rigour that 
could be compared directly to other approaches to higher education. 
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Figure 2.4: Balancing face-to-face contact with materials. Adapted from 
Rowntree (1997). 


This approach has had various names, and there have been changes over time. 
The OU for some time used the label of “Supported Open Learning”. This empha- 
sised the support and contact component, as differentiating the approach from a 
correspondence based approach, and acknowledged the core role of the tutor in 
providing greater resilience and the ability to cope with the individual differences 
in the needs of the learners. A further label that has been given is of “Materials- 
based learning’ (Rowntree, 1997). This emphasises the use of materials in place of 
contact that is designed to share knowledge. The concept of material based learn- 
ing is one end of a continuum (see Figure 2.4 - based on the introductory figure 
given by Rowntree, 1997) that goes from traditional classroom based instruction 
through to learning at a distance from materials. Note that in Rowntree’s model 
the contact does not fall to zero in how he considers materials, i.e. he is assuming 
that some contact will always be part of the teaching process. 

The balance between a structured approach to the design of teaching and the 
dialogue and freedom to consider ideas was considered by Michael Moore dur- 
ing his time at the OU in the 1970s and 1980s and was described later under 
the label of Transactional Distance (Moore, 1993). This contained a rethink- 
ing of the distance element of distance education away from physical distance 
to measure instead how close the approach was to being able to transact the 
education through dialogue. In this model low transactional distance would 
be achieved through free-flowing dialogue between the learner and the educa- 
tor which gave considerable autonomy to the learner. Higher transactional dis- 
tance would be constrained provision of ideas in a tight structure that did not 
allow for the learner to take control. In conventional teaching, high transac- 
tional distance would occur in a content heavy lecture that offered little expec- 
tation of questions, low transactional distance in a one-to-one personal tutorial 
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that allowed open discussion. The theory did not make any judgement as to 
which was the better approach, rather it highlights for distance education that 
the challenge is to develop ways to vary the transactional distance and under- 
stand how these can operate in combination to benefit the learner. 

For the OU the challenge of course design recognised that the learner needed 
to be able to be supported so that materials guided them to achieve the origi- 
nal intentions stated in the Planning Report to address the “limited oppor- 
tunities for education, determined by social, economic and political factors” 
(HMOS, 1969, p.2). 


The role of evidence 


The fourth component of the Beyond Prototypes model is evidence. This, per- 
haps more than anything, is how The Open University differentiated itself from 
other universities when it was formed and it has remained a key part of the 
OU’s approach. Evidence of the impact of teaching was needed to show that new 
approaches resulted in a university that met quality requirements. This meant 
developing understanding of the processes of the university and how they could 
be justified, monitoring the performance of its students, and finding out what 
was the students’ own view of their experience. This last component was very 
much an innovation and was supported through a process that included large 
scale surveys, which in turn influenced the development of the National Student 
Survey adopted in the UK (Richardson, Slater and Wilson, 2007). The Institute 
of Educational Technology has played a central role in the gathering of relevant 
research, in the application of that research to cases inside and outside the OU 
and in addressing the challenge of communicating an evidence base. 

An evidence-based approach to decision making has often been advocated. 
Arguably this has been less apparent in education and in particular higher educa- 
tion where status and tradition can dominate over innovation and evidence. The 
revolution provided by the OU meant that it pushed against that tradition and 
needed to show that it could function, and meet similar quality measures to other 
universities, while working with a broader body of students and at a larger scale 
than other universities. The starting points rethought the approach away from 
contact-based teaching to one where content and support could operate together 
to provide the learners with all that they needed. As discussed in McAndrew 
(2010), a useful metaphor for the time (1960s) was of being able to support the 
lighthouse keeper who was physically remote from many resources, though with 
access to communications technology in the form of television and telephone. 


Data wranglers 


One of the challenges of gathering large amounts of data is the time and exper- 
tise required to interpret it. A focus on understanding the impact of different 
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media on students had been part of the remit of IET first through an Audio Vis- 
ual Media Research Group and then a more targeted Programme on Learner 
Use of Media, established in 1990 (Weinbren, 2015). In 2009 the focus moved 
from considering media to looking at data itself. At that time the Student Sta- 
tistic and Survey Team had achieved a partial success in making available sum- 
marised date and then giving online access to data for deeper inspection. This 
was supported by training and opportunities to request further analysis. Even 
with careful planning the reality was that use of data was not uniform across 
the OU and there was a perception that data was more often used to consider 
the local concerns of an individual module rather than the priorities of the 
university or faculty. While the module will always be a key element of experi- 
ence, common challenges such as retention and student performance required 
a more coordinated approach. 

An academic team was created that would work in a targeted way both 
within faculties and across faculties on shared priorities. The focus on data was 
recognised in the chosen name for the team, Data Wranglers (Clow, 2014), and 
in initial targets, to make sure that clearly communicated reports that high- 
lighted the key metrics needed by all parts of the university were provided. 
These developed over time to also include comparisons between different parts 
of the university and easy to interpret indicators of relative performance, one 
example being thumbs-up/thumbs-down signs to quickly show differences 
between different parts of the university for measures of student performance 
and satisfaction. 

A role that developed out of the Data Wrangling approach, and linked to 
their typical position as academics also carrying out research inside and outside 
the university, was the provision of thematic reports initiated in 2016 (Rienties 
et al., 2016). These reports have covered a mix of OU-specific topics, such as 
understanding the different registration points a student passes through while 
studying, and wider issues, such as the role of summative and formative assess- 
ment and approaches to feedback. 


Summary 


In this chapter the Beyond Prototypes model, developed to analyse the pro- 
cesses of innovation in technology-enhanced learning and consider what can 
help enable successful TEL innovation, has been applied to one of the larg- 
est on-going educational technology projects, The Open University itself. The 
chapter has shown how the four aspects of innovations in the Beyond Proto- 
types model (the TEL complex, persistent intent, bricolage and evidence) can 
be successfully applied to The Open University. McAndrew has argued that 
complexity was a fundamental aspect of starting The Open University. In going 
beyond a correspondence approach, the original concept of “the University of 
the Air’, with broadcasting playing a key role, was viewed very broadly and in 
developing an ambitious and complex structure for the OU, its planners learnt 
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from relevant work in the US. So the first section considers how complexity 
was addressed by those working to bring The Open University together as they 
thought through how teamwork, and the role that educational technology 
needs to play could work in practice. 

A clear example of persistence, the second component of the model, is con- 
tinuing open entry, requiring support for entry into study. In discussing this 
aspect of the university, OU data was drawn on which shows it is only at the 
beginning of a student’s study trajectory that their likely success can be judged 
from their background and or earlier achievements. Once a student’s journey is 
underway the best predictor of success is previous successful study in the OU. 

The chapter has also illustrated how the OU had to take an experimental 
approach, (applying the third factor, bricolage) from the very beginning; partly 
to address the complexity challenge: in working out how to operate at such 
scale; how to structure itself; how to remain innovative in a changing world 
and how to make best use of technology for educators and for students. Exam- 
ples include providing online virtual learning environments, online conferenc- 
ing and remote connections, and making changes to assessment. One of the 
innovations in the design of OU courses is that content is closely supported by 
feedback to the learner, and in this context, the chapter has shown how feed- 
back operates both through the design of the learning experience and in how 
support is provided. 

Evidence is the fourth aspect of the framework and this is shown to be a 
particularly important part of the OU’s approach and significant in the distinc- 
tiveness of the OU. In this respect IET work has been key and has collected evi- 
dence on student learning and attitude from the very beginning. One example 
of the role of evidence is the use of ‘Data Wranglers: 


A model for now 


The Open University was intended to be different; to try out new ideas and go 
beyond prototypes to operate at scale. The OU of fifty years ago was a challenge 
to a model of limited access to university. Now such a model seems even more 
unsuited for the situation we are in. Wider access to education has not led to 
equal access with participation from those disadvantaged remaining behind on 
several different measures such as POLAR, and ethnicity. Passing on costs to 
the learner in the form of expensive loans has also created pressure on those 
who are already earning, in particular leading to a fall in the numbers of those 
studying part-time. 

Taken together this means that there is a greater need to consider the ways in 
which we can effectively operate a larger scale, lower cost, university system for 
those who have a wider background of prior qualifications. This is the current 
challenge facing the whole sector. It also characterises the challenge that the 
OU has faced for 50 years. 
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Many aspects of this challenge, and how it is being addressed, are discussed 
in the following chapters. In particular, the next chapter, Chapter 3, considers 
the future of teaching and learning at scale. 
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CHAPTER 3 


Teaching and Learning at Scale: Futures 


Rebecca Ferguson 


In this chapter, Rebecca Ferguson considers recent work toward the vision “Teams 
can successfully teach any number of students at a distance’, showing how a sub- 
stantial body of TEL research work can be built up over time, responding to changes 
in society. In particular, she demonstrates how continuing work towards this vision 
relates to the emergence of massive open online courses (MOOCs) and, more 
broadly, to teaching and learning at scale. She shows how the different elements of 
the Beyond Prototypes framework, and its emphasis on bricolage and persistent 
intent, can be used to support the development of a research agenda that supports 
practice worldwide. She also looks at current and future work in this area, identify- 
ing key areas where work is still needed - learning design, educator teams, widening 
access, approaches to assessment and accreditation, and new forms of pedagogy. 


Introduction 


In 2015, world leaders attended a United Nations Sustainable Development 
Summit in New York, where they formally adopted a new sustainable develop- 
ment agenda, setting goals to transform our world. One of these goals is quality 
education, and a target to be achieved by 2030 is to ‘ensure equal access for all 
women and men to affordable and quality technical, vocational and tertiary 
education, including university: 
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This is an ambitious goal, particularly in the light of the numbers involved. 
Three years before the world summit, Daniel looked at the rapidly increasing 
demand for education in a world where, today, more than a quarter of the pop- 
ulation is aged under 15 (The World Bank, 2018) 


there are 165 million people enrolled in tertiary education [...] Projections 
suggest that participation will peak at 263 million [...] in 2025. Accom- 
modating the additional 98 million students would require more than four 
major campus universities (30,000 students) to open every week for the 
next 15 years. (Daniel, 2012) 


This hasnt happened. Major campus universities arent opening every couple 
of days. However, a phenomenon that grabbed the world’s attention seemed to 
be the answer - massive open online courses (MOOCs). What had begun as 
a Canadian experiment in teaching and learning (Cormier, 2008), suddenly 
hit the headlines. These online courses, often from top-ranked universities, 
were openly available, which typically meant that they could be accessed free 
of charge by unlimited numbers of learners. The New York Times declared 2012 
The Year of the MOOC, observing that ‘more than 150,000 signed up for Dr 
Thrun’s “Introduction to Artificial Intelligence” last fall, starting the revolution 
that has higher education gasping’ (Pappano, 2012). 

The stated aims of early MOOC providers related to the challenge of extend- 
ing access to education. The original MOOCs had been designed to increase 
participation in lifelong learning (McAuley, Stewart, Siemens, & Cormier, 
2010). The mission statement of the Coursera platform was ‘to empower people 
with education that will improve their lives, the lives of their families, and the 
communities they live in. The founder of the Udacity platform was 


against education that is only available to the top one per cent of all stu- 
dents. I am against tens of thousands of dollars of tuition expenses. I am 
against the imbalance that the present system brings to the world. I want 
to empower the 99 per cent’ (Leckhart & Cheshire, 2012). 


World-class teaching available, free of charge, for everyone, everywhere in the 
world? It seemed too good to be true - there had to be a catch. 

In fact, there were several. The most widely publicised was the drop-out 
rate. The percentage of enrolled students who completed a MOOC varied 
widely but, typically, about seven students of every eight who signed up for a 
course did not reach the end (K. Jordan, 2015). This isn’t necessarily a prob- 
lem - a MOOC can be regarded as a resource, like a newspaper, that most 
people will never complete (Downes, 2014). However, the wide variation in 
completion rates - from a low of less than one student in forty to a high of 
more than half of students - suggested that there were other factors at work 
(K. Jordan, 2015). 
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There were other problems as well. Problems with funding - good quality 
MOOCs turned out to be expensive to produce. Problems with revenue gen- 
eration - even the original low-budget MOOCs based on connectivist princi- 
ples required server space and staff time. Problems with pedagogy - enormous 
class sizes hit the headlines but offered no clear benefits for learners. Problems 
with technology - even in the affluent west, not everyone has access to per- 
sonal computing devices and the Internet. In fact, there were problems to be 
addressed that related to every element of the Beyond Prototypes framework: 
policy context, environment, funding, revenue generation, educator com- 
munity, technology community, learner community, researcher community, 
ecology of practices, technical context, pedagogy and technology (Ferguson, 
Sharples, & Beale, 2015; Scanlon et al., 2013). 

It was apparent that MOOCs - or, more broadly, learning at scale - didn't 
offer a magic pill that could quickly remedy a chronic shortage of access to 
higher education. In fact, it wasn't clear that this was the vision that was guid- 
ing the development of most MOOCs. Universities were more interested in 
using them to enhance the reputation of the institution, to develop staff skills, 
or to add value to accredited courses (White, Davis, Dickens, León Urrutia, & 
Sanchez-Vera, 2015). In addition, studies of the first wave of MOOCs suggested 
that learners were mainly people with prior experience of higher education, 
and found little evidence that MOOCs were widening participation for those 
distanced from education (Cannell & Macintyre, 2014). 

From the perspective of The Open University, MOOCs had the potential to 
align with its mission to be ‘open to people, places, methods and ideas. They 
also promised to be a way of achieving its goals of promoting educational 
opportunity and social justice. The vision that “Teams can successfully teach 
any number of students at a distance’ was already an intent that the university 
had been pursuing persistently for decades. Making the connection with this 
intention meant that researchers at The Open University were able to build on 
previous work, using MOOCs as a way of working towards that goal. 


Persistent intent 


“Teams can successfully teach any number of students at a distance’ was ini- 
tially a vision for one university. The scope of the vision expanded as increasing 
numbers of learning institutions worldwide adopted online and distance learn- 
ing. The possibilities opened up by new technologies led to further expansion, 
taking into account the growth of open educational resources, social media, 
and open learning on platforms such as YouTube and iTunesU. 

This experience meant that researchers at the OU did not approach MOOCs 
as a completely new phenomenon. Instead, they were able to engage in a pro- 
cess of bricolage, bringing in their experience of investigating open learning, 
large-scale citizen science projects, and the use of data to support learning and 
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teaching. The new feature offered by MOOCs was not that they were open, or 
online, or courses. The new feature was that they were massive. 

‘Massive needs some explanation in the context of a university with decades 
of experience of teaching many thousands of students on the same course at 
the same time. MOOCs were massive in that they were designed to be accessed 
by unlimited numbers of students. In some cases, tens of thousands enrolled, 
and in some cases only tens, the ‘massive’ label was determined by design 
rather than by enrolment figures. Platforms and media tended to present this 
scale in terms of potential economic benefits. Large numbers of people could 
be exposed to the same curated set of resources and gain access to teaching 
materials assembled by well-known universities. Fewer trained educators 
were, potentially, required to educate greater numbers of learners (Sharples & 
Ferguson, 2014). 

This brought a new perspective to the vision that ‘teams can successfully teach 
any number of students at a distance’ Previous research in this area had focused 
on distance education at university level, in a context where students either paid 
for their education, or had that education paid for. Taking on more students 
meant taking on more staff to act as tutors, mentors, facilitators, assessors, and 
examiners. In a setting where learners could complete a course free of charge, 
new approaches to pedagogy were required, as well as attention to learner and 
educator communities, and the ecology of practices around MOOCs. 


Pedagogy at scale: conversational learning 


Many approaches to teaching and learning - such as sports coaching and per- 
sonal tuition - are designed to work with a limited number of learners and do 
not scale. Only a few pedagogies can be used with cohorts that may range from 
tens of learners to tens of thousands. Lecturing is one such approach. As long 
as learners are able to hear the lecturer and see the presentation, the experi- 
ence is broadly similar for any number of students. However, students gain little 
by being part of a large cohort and are unlikely to be able to ask questions to 
increase their understanding. 

Another approach is the OU’s model of supported distance learning (Price 
& Petre, 1997). As learners work through their studies, they are supported by 
many teams of academic and administrative staff, as well as by associate lectur- 
ers who provide personalised study support and feedback. This model scales 
successfully, and opens up opportunities for collaborative and social learning, 
but providing this level of support is expensive. 

An alternative approach is conversational learning. This not only works at 
scale, the conversations become richer as more learners are involved. 

The FutureLearn MOOC platform was set up in 2012-13, based on a con- 
versational learning pedagogy that draws on decades of work within the OU. 
Learning through conversation relates to the theory of how learning takes 
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place that was developed by Pask (1976). Pask provided a scientific account 
of how interactions enable a process of coming to know by reaching mutual 
agreements. This is more than an exchange of knowledge - it is a process in 
which participants share and negotiate differences in understanding with the 
aim of constructing new knowledge and reaching agreements. These conversa- 
tions may involve other learners or may take the form of internal reflections. 
In order for either to happen, ‘learners must be able to formulate descriptions 
of their reflections on actions, explore and extend those descriptions, and carry 
forward the understanding to a future activity’ (Sharples & Ferguson, 2019). 

Laurillard (2002) built on Pask’s work when she developed her influential 
Conversational Learning framework. Discussions take place at two levels: 
actions and descriptions. Learners need to agree on clear goals and objectives 
at both levels. 


e At the level of actions, discussion relates to a practical activity or model 
of the world. Learners ask ‘how’ questions, sharing their experiences and 
interpretations. 

e At the level of descriptions, learners ask ‘why’ questions, putting forward 
and questioning interpretations in order to reach agreement. 


The educator plays an important role throughout the process: suggesting goals, 
designing relevant activities and models, facilitating discussions, and encour- 
aging reflection. 

These understandings of how learning takes place, and how it can be enriched 
at scale, were built into the FutureLearn platform from the beginning and con- 
tinue to inform technical and organisational developments. For example, a dis- 
tinctive element of the platform is that, except in the case of assessment, every 
piece of learning material has an associated area for conversation. Because con- 
versation takes place alongside content, it becomes part of the learning materi- 
als, rather than a separate activity that requires extra work to access and is likely 
to be disregarded. Some steps are designed as discussions, providing opportu- 
nities for learners to explore differences in conception and reach agreements. 
These discussions can be set up by educators in ways that encourage learners to 
share perspectives, synthesise new knowledge, and reach agreements. 


Pedagogic research community 


Using decades of research to support the development of a platform that cur- 
rently has more than ten million registered learners worldwide was a very posi- 
tive outcome for this line of work but was not the end of the process. More 
research was needed - and with learning taking place on a grand scale, a larger 
pedagogic research community was needed to support and develop the process. 
This requirement led to the formation of the FutureLearn Academic Network 
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(FLAN), an association of people carrying out MOOC-related research at 
FutureLearn’s partner institutions (by mid-2019, FutureLearn had more than 
160 partners around the world, including universities, specialist organisations, 
and centres of excellence). 

A 2016 review of research work by a sub-section of these - FutureLearn’s UK 
university partners - covered 109 publications and identified priority areas for 
future research. These included: 


e develop educator teams 

e identify and share effective learning designs 

e support discussion more effectively 

e widen access 

e develop new approaches to assessment and accreditation 
e develop appropriate pedagogy for learning at scale. 


The following sections consider each of these research areas and relate them 
to the vision that ‘teams can successfully teach any number of students at 
a distance. 


Develop educator teams 


The educators responsible for leading work on MOOCs are typically university 
faculty members. Despite their expertise in research and in face-to-face teach- 
ing, they may have little or no experience of teaching at a distance, online, or at 
scale. Nevertheless they are faced with a series of difficult educational decisions 
and need research evidence that can help them to make decisions about issues 
as diverse as assessment, engagement, motivation, design, and accreditation. 

While tackling these questions, educators are also adapting to a new role. 
They are likely to become increasingly aware that their traditional role has been 
‘unbundled’ and that the tasks of designing a course, locating resources, pre- 
senting a course, assessing students’ work, and supporting the students are car- 
ried out by a variety of people with different skills. Research-based evidence 
has the potential to help them make that move from lone educator to part of a 
distributed team. 

Broadly speaking, teaching roles on FutureLearn fall into three broad catego- 
ries: educators, mentors and collaborators (Papathoma, 2019). 


e Educators typically work in academic roles at a university. Whatever their 
substantive job, they are likely to be involved in a MOOC because of their 
subject-matter expertise. 

e Mentors are sometimes described as facilitators. This is a role that is often 
assigned to doctoral students. There may be an assumption that mentors 
have less experience and less subject matter expertise than educators. They 
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are likely to be involved ina MOOC in order to engage in learner discussion 
in some way. 

e Collaborators support learning activity but are not expected to be directly 
involved with learners. Collaborators are typically not faculty members but 
are likely to work in academic-related jobs as, for example, learning design- 
ers, librarians, or managers of learning teams. 


Each group brings different skills and perspectives to their teaching role on 
the MOOC and faces different sets of challenges. Ideally, individuals in each 
role work together seamlessly, drawing on each other's expertise. However, 
this way of working does not come automatically and is difficult to develop 
when time is limited, and team members do not necessarily meet each other 
or interact frequently. 

Teams involved in course development need access to both practical and 
academic expertise. They must be aware of the team’s responsibilities and the 
constraints under which both it and its individual members are working. To 
develop a course effectively, teams have to be willing to agree ways to negotiate 
these constraints. This means that professional learning ‘is a critical component 
of the ongoing improvement, innovation and adoption of new practices that 
support learning at scale’ (Papathoma, Ferguson, Littlejohn, & Coe, 2016, p1). 

Successful teams involved in teaching on MOOCs (Papathoma, 2019) give 
explanations for aspects of teaching, developing shared vocabulary and under- 
standing of the teaching process. They reflect on the process and explain aspects 
to each other, building new, shared knowledge that can be structured and 
recorded to support the development of subsequent MOOCs. If these oppor- 
tunities are not built into the process, individuals have to deal with uncertainty 
and are forced to spend their time solving problems, searching for individuals 
with relevant expertise, and looking for helpful examples of previous practice. 


Identify and share effective learning designs 


Previous practice in learning design is an aspect of professional knowledge that 
can be shared between academic departments and institutions. As Chapter 7 
will explain in more detail, learning design provides a way of sharing teaching 
ideas in order to improve student learning, helping educators to become more 
effective in their preparation and facilitation of teaching and learning activities. 
Design patterns provide a way of showcasing successful learning activities and 
design innovations, as well as making clear which approaches do not work. 
Design patterns also provide ways of sharing solutions to problems that are 
commonly encountered when designing MOOCS. For example, Wintrup, 
Wakefield & Davis (2015) note that dropout is a concern with MOOCs, and 
that there is a need to identify measures that can be put in place to reveal 
what aspects of a course engage learners, and how particular activities engage 
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different types of learner. Part of the solution might be to make an up-to-date 
recap of each MOOC available at any time so that new joiners can catch up 
with others (Nazir, Davis, & Harris, 2015). Another part of the solution might 
be to minimise distractions that do not support design objectives by organising 
resources, enabling creative expression in tasks, automating mundane tasks, 
supporting scale and sustainability, and focusing on learning (Celina, Kharuffa, 
Preston, Comber, & Olivier, 2016). 

A review of published research from FutureLearn partners in the UK (Fergu- 
son, Coughlan, Herodotou, & Scanlon, 2017; Ferguson, Herodotou, Coughlan, 
Scanlon, & Sharples, 2018) identified design patterns that appeared promis- 
ing or had proved successful in the context of MOOCs. Some of these were 
brought together by Wintrup and her colleagues: providing guidance about 
ways to apply new knowledge to ‘real world’ problems can be helpful in deep- 
ening and sustaining understanding and promoting creativity, including and 
eliciting learners’ own ideas and projects is also a way of developing greater 
involvement, games provide a useful way of introducing difficult concepts to 
learners (Wintrup et al., 2015). 

Liyanagunawardena, Kennedy and Cuffe (2015) organised a series of work- 
shops to explore MOOC design principles. In particular, they considered the 
challenge of promoting peer discussion and interaction when the size and 
diversity of a cohort and its patterns of participation mean that discussions 
become difficult to navigate and are likely to remain superficial. They identi- 
fied seven design narratives that captured and interpreted the experience of 
MOOC designers. They then drew on these narratives to create design patterns 
that offered solutions to challenges commonly encountered when designing 
MOOCs. For example, the ‘Look and Engage’ design pattern provides a solu- 
tion to the problem of ‘How to structure peer communication and collabo- 
ration to support the sharing of ideas to stimulate meaningful dialogue and 
interaction among large, diverse groups’ (p12). The pattern involves creating 
‘an individual collaborative task around a digital artefact to stimulate meaning- 
ful dialogues among large, diverse groups’ (p10). ‘Look and Engage’ draws on 
three design narratives that deal with scaffolding interaction, easy co-construc- 
tion, and sharing views. 

Hatzipanagos (2015) went a step further, not only identifying design pat- 
terns used in MOOCs but also relating these to patterns used elsewhere, begin- 
ning to build the links between designs that can reveal underlying similarities 
between courses. For example, he related the pattern ‘Computer-mediated 
communication media (fora)’ to a previously identified ‘crowd bonding’ pat- 
tern, summarised as ‘forming discussion groups to facilitate interaction for 
learning: By making connections in this way, he demonstrated ways in which 
patterns could be used to access previously developed guidelines, advice, and 
practical examples. 

Work on identifying and sharing effective learning designs demonstrates 
how TEL innovations are built over years, pulling together available resources 
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in a process of bricolage. Educators around the world face similar challenges. 
How can dropout be reduced? How can peer communication be structured to 
support meaningful dialogue? They explore solutions in different contexts with 
different student populations. Researchers identify and publish solutions, but 
these are distributed across a wide range of literature and not easy to access. 
Research groups, such as the FutureLearn Academic Network, then work to 
pull together the challenges and solutions. 

A different stream of work, on learning design, provides a way of structuring 
these challenges and solutions as design patterns that can be brought together 
and shared. At the same time, work on opening up research and education 
makes it increasingly acceptable to share work openly, rather than restricting 
it to journals hidden behind paywalls. As a result of these separate strands of 
work, the European bizMOOC project was able to create and openly share 
its MOOCBook, which brings together 50 key lessons, 25 key recommen- 
dations, and 20 good practices derived from extensive empirical research 
(BizMOOG, 2019). 


Support discussion more effectively 


One of the challenges when designing MOOCs is how to provide effective 
support for discussion. This is crucial when a course design is based on con- 
versational learning. It is also more broadly applicable. The earliest distance 
education courses, which relied on published or posted material, offered little 
or no opportunity for discussion. They were based on a transmission model 
of education, which assumes an existing body of knowledge that can be trans- 
ferred from one person to another, with assessment providing opportunities to 
check that the transfer has been completed successfully. 

One problem with relying on a transmission model alone is that separat- 
ing learners and teachers in time and space creates a space of potential mis- 
understanding, or ‘transactional distance’ (Moore, 1973, 1993) as noted by 
McAndrew in Chapter 2. The possibilities for reducing this space for misun- 
derstanding increased when new communication technologies made it pos- 
sible for distance learners to interact with each other and with educators. Most 
of this interaction was asynchronous, with no expectation that participants in 
a discussion would all be engaging at the same time. These new technologies, 
such as forum discussion, opened up opportunities for learners to engage in the 
active construction of knowledge together, as well as working to understand 
existing content. 

However, online and offline interaction are not the same. Some types of inter- 
action that commonly take place in the classroom are much rarer online, even 
though they shape learners’ expectations of how interactions with teachers will 
take place. For example, a typical exchange between teacher and student in a 
face-to-face classroom involves initiation, response, and follow-up (Sinclair & 
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Coulthard, 1975). The teacher asks a question, students recognise that this is 
not a request for knowledge but a test of their own knowledge, one or more of 
them responds, and the teacher evaluates or extends those responses. Learners 
who expect educational discussions to take this form often struggle to see the 
value of conversations in forums or MOOCs where there is no teacher evaluat- 
ing or extending their contributions. 

Conversations in online environments share several characteristics that dis- 
tinguish them from face-to-face conversations. Some of these are negative. It 
isnt necessarily clear who is taking part in the conversation, who is reading 
without posting, how much of the conversation any one person has read, and 
in what order they encountered it. Some students find online discussion intimi- 
dating and are nervous about contributing (Ferguson, 2010). On the positive 
side, asynchronous communication offers time to reflect before contributing, 
a transcript of the entire discussion, interaction at a distance, and opportu- 
nities to share direct quotations, references, and links to external resources 
(Ferguson, 2009). 

With research on the benefits and challenges of social learning and online 
discussions already in place, MOOC researchers were able to focus on the 
practicalities of involving people in discussions. As a significant percentage of 
learners in any MOOC will not have studied online before, it’s important to 
encourage good practices through providing guidance and examples. Good 
practices include the use of inclusive language, treating different viewpoints 
with respect, and encouraging social interaction that will support learn- 
ing (Murray, 2014; Wintrup, Wakefield, & Davis, 2015; Wintrup, Wakefield, 
Morris, & Davis, 2015). 

More detailed work is now being carried out to investigate ways of sup- 
porting interaction. The large number of comments posted in many MOOC 
discussions can disguise the fact that relatively few meaningful conversations 
are taking place. Many learners receive no response to their comments, which 
means they are unable to take part in the development of shared understand- 
ing. To some extent, the likelihood of response is based on the time of post- 
ing and on the nature of discussion. However, keyword analysis has shown 
that there are also linguistic factors at work. Posts that receive responses are 
often phrased as questions. They use non-specific pronouns such as ‘anybody’ 
or ‘anyone. They also hedge rather than making definitive statements, using 
words such as ‘perhaps’ and ‘seems’ (Chua, 2018). Work like this can be used 
to support guidelines for MOOC participants, and the models of interaction 
provided by MOOC educators. 

Adult learners and educators typically have extensive experience of how 
educational interactions work in a face-to-face environment but may have 
little or no experience of how this can be done effectively online. Different 
strands of work, relating to appropriate pedagogies, learning design, and 
discussion can be combined to develop best practice, based on research 
and experience. 
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Widen access 


Best practice is needed, because MOOCs do not currently provide learning 
opportunities that are suitable for everyone. Although we are making good 
progress towards the goal, “Teams can successfully teach any number of stu- 
dents at a distance, more work is needed on inclusion and accessibility. This is 
particularly important, given that the original vision for MOOCs was that they 
could open up high-quality education to groups who had previously had no 
means of accessing it. 

One aspect of widening access is reaching learners in areas that currently 
offer few opportunities for higher education. However, online learning is not 
necessarily the answer when some four billion people around the world still do 
not have access to the Internet. Global access would need to include regions that 
have poor infrastructure, low digital capability, unreliable electricity supplies, 
limited digital capability, and that currently lack capacity to train all teachers 
to a high standard. It would also have to take on the challenge of providing 
equal access to resources in countries that have multiple official languages and 
diverse ethnic communities (Littlejohn & Hood, 2018). 

These are not new problems. Two long-term projects have investigated some 
potential solutions. The Teacher Education in Sub-Saharan Africa (TESSA) 
project launched in 2005 at the request of teacher education institutions 
throughout the region, and its success led to the creation of a sister programme, 
TESS-India. Both projects have at their heart resource banks of materials to 
support teacher education. These take into account the cultural diversity of the 
regions involved, and are available in multiple languages, in both printed and 
digital formats, online and offline (Wolfenden, 2008). 

The TESS-India MOOC, which ran on the EdX MOOC platform, was able 
to build on more than a decade of research and experience. It was designed 
to introduce the idea of open educational resources (OER), and particularly 
the TESS-India OER to teacher educators. The rationale for working with this 
group was that these professionals have the opportunity to initiate significant 
changes in teaching and learning across the region if they have access to rel- 
evant training and resources (Stutchbury, 2016). 

Due to the low bandwidth for internet connections across much of India, 
the TESS-India MOOC was not run wholly online. It included weekly contact 
classes in all the project’s target states. This face-to-face contact supported the 
development of local communities of practice (Stutchbury, 2016). As a result, 
community members were able to support each other to extend and implement 
what they had learned. The learning design was also successful in supporting 
retention. The MOOC ran in English in late 2015 with over 10,000 people reg- 
istered. Of these, 51% completed the course, with 81% of completers from the 
states where face-to-face support had been available. A second iteration the 
following year in Hindi attracted over 33,000 participants, of whom 52% com- 
pleted the course (Wolfenden, Cross, & Henry, 2017). 
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Even when MOOCs are available in a relevant language, culturally appropri- 
ate, and suited to available technology and infrastructure, problems remain. 
MOOGCs offer few opportunities for personalised interaction with an educator, 
which means that most people who sign up for MOOCs are responsible for reg- 
ulating their own learning. Self-regulation requires a set of skills that take time 
to develop - many of which students are typically not expected to demonstrate 
until university level. These skills include time management, help seeking, stra- 
tegic planning, goal setting, reflection, and self-evaluation. Most people initially 
find it challenging to apply this set of skills to their learning. This is particularly 
true when they are used to a teacher doing much of this work for them and they 
have not been supported to develop these skills for themselves. 

Some of the variety of learning behaviours of MOOC participants relates 
to their ability to regulate their own learning (Littlejohn, Hood, Milligan, & 
Mustain, 2016). For example, those who score low on a measure of self-regu- 
lated learning focus their goals on traditional performance measures such as 
passing assignments and completing the course, while those who score higher 
are more interested in developing relevant knowledge and expertise. Partici- 
pants with low scores are likely to be focused on the MOOC and its require- 
ments, while those with higher scores for self-regulation are more interested in 
how they will use what they have learned (Littlejohn et al., 2016). This suggests 
that if teams want to be able to ‘successfully teach any number of students at a 
distance’ then they need to make sure that those students are prepared to take 
an active role in regulating their own learning. 

They also need to make sure that the courses they offer are accessible. There 
are four key aspects to MOOC accessibility: learning design, technical ele- 
ments, user experience, and overall quality (Iniesto, McAndrew, Minocha, & 
Coughlan, 2019). Universal design, an approach that considers how to meet 
the needs of all learners through design, provides a helpful starting point 
(McGuire & Scott, 2006). Technical accessibility can be shaped using the Web 
Content Accessibility Guidelines (WCAG) published by the World Wide Web 
Consortium, the main international standards organisation for the Internet. 
These guidelines focus on whether material is perceivable, operable, under- 
standable, and robust. From a user-experience perspective, the activities within 
any MOOC need to be feasible for learners with a range of accessibility needs. 
More broadly, a quality audit can be used to scrutinise accessibility in terms 
of staff and student support, as well as curriculum design, course design and 
delivery, and assessment (Iniesto et al., 2019). Accessibility more generally in 
educational technology is discussed in the next chapter. 


Develop new approaches to assessment and accreditation 


Assessment and accreditation have a role to play in widening access to edu- 
cation, so they are important aspects of supporting learners in large-scale 
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environments. MOOC participants who already have degrees or even post- 
graduate qualifications may enjoy informal study for the love of learning, but 
some form of accreditation for study is particularly important for those who 
do not already have the qualifications that will help them to acquire a job or 
develop a career. 

In order to cover costs, many MOOC providers now charge for credentials 
and certification and do not make the entire learning experience freely avail- 
able. This modified approach retains an open element - MOOC participants 
have the opportunity to study material without charge - but credit for that 
study comes as an optional extra that requires payment. Another shift away 
from openness is that the major MOOC platforms now offer courses that are 
only available to those who pay, challenging notions that an open course offers 
access for all or free education (Littlejohn & Hood, 2018). 

Following a review of research by MOOC-providing institutions across the 
UK, Ferguson and her colleagues (2017) recommended that MOOCs should 
provide transparent information about accreditation to learners, institutions 
and employers; that MOOC providers should consider ways of supporting 
credit transfer; and that providers should also supply guidance to MOOC 
learners for recognition of non-formal learning, as awareness of the available 
options is currently limited. 

Badging is one way of supporting the route from assessment to accreditation 
(Law & Law, 2014) without a charge to the learners. Open badges have two 
elements: an online image containing a hyperlink to course criteria, and online 
evidence that these criteria have been met (Cross, Whitelock, & Galley, 2014). 
Badges can be used as incentives to continue study, as a way of marking pro- 
gress, as an informal means of accreditation, or as staging posts on the journey 
to more substantial learning goals (Hauck & MacKinnon, 2016). They can be 
used to reward achievement at marked points on a learning journey, to reward 
effort in terms of hours put in or activities completed, or they can reward 
exploration and deeper learning (Cross & Galley, 2012). Not simply a means 
of accreditation, badges can function as motivator, meaning maker, signifier of 
learning objectives, low-cost or low-effort option. They can be used as a way of 
valuing certain forms of engagement, a symbol of identity, a means of associa- 
tion, or an element of empowerment. They also have roles to play in encourag- 
ing engagement and limiting withdrawal (Hauck & MacKinnon, 2016). 

Closely associated with accreditation is assessment, which plays a crucial 
part in learning and teaching. Expert feedback is a valuable part of the learn- 
ing process, but it takes effort to produce. Skilled assessors come at a cost and 
their availability is limited. Together, these factors make assessment a particu- 
larly challenging aspect of learning at scale. Producing high quality feedback 
is not an activity that scales easily. MOOCs therefore need to make use of 
the full range of computer-based assessment options. These currently include 
selected responses (such as multiple-choice questions); constructed responses 
(in which learners construct their own responses); essays and short-answer 
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questions; peer assessment; and online tools that can be used to showcase 
work, including e-portfolios, blogs and wikis (Jordan, 2013). None of these 
options is new, but as these forms of assessment play an important part in 
learning at scale, more work is needed to explore how they can best be imple- 
mented and validated. 

Peer review offers a way of providing feedback on student work that can- 
not be assessed automatically. It also provides a learning experience for the 
reviewer, who has to think carefully about the criteria and how they are applied. 
However, work on supporting and structuring peer review at scale in an inter- 
national context is still in its early stages (Meek, Blakemore, & Marks, 2016; 
O'Toole, 2013). 

Another form of assessment that is commonly used in MOOCs is the mul- 
tiple-choice quiz. These can be helpful in providing formative assessment 
and helping learners to assess their understanding, as long as educators are 
skilled in their design and implementation. However, unless multiple-choice 
questions are underpinned by extensive question banks, there is the danger 
that answers will be shared online, making them unsuitable for summative 
assessment. There is a need to build on what we already know about e-assess- 
ment (Jordan, 2013), so that appropriate forms can be built into pedagogy 
at scale. 


Develop appropriate pedagogy for learning at scale 


As learning at scale is taken up more widely and in new contexts, appropri- 
ate pedagogies are required to support this work. Conversational learning is 
one pedagogy that has already been incorporated within MOOC teaching and 
MOOC platforms, but there are other pedagogies still to be explored, including 
adaptive teaching, experiential learning, game-based learning and inquiry-led 
learning (Sharples & Ferguson, 2019). 

One reason for developing new pedagogies is the increasing use of MOOCs 
to support workplace training, job readiness, and continuing professional 
engagement. Workers are looking for forms of personalised learning that align 
with their specific learning needs. They also need to develop skills and strate- 
gies that enable them to deal with the ill-structured problems under various 
levels of uncertainty that they are likely to encounter in their workplaces (Lit- 
tlejohn & Hood, 2018). 

Pedagogies that have been developed in other contexts and could be adapted 
for use at scale include: social learning to share workplace knowledge, coached 
team learning to develop and practise skills, case-based learning for problem 
solving and decision making, experiential learning to capture and reflect on 
shared experience, and competency-based learning to achieve and demon- 
strate mastery. Once again, there are opportunities to build on extensive previ- 
ous work and to make use of recognised good practice. 
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Conclusion 


“Teams can successfully teach any number of students at a distance’ is one of 
the visions that has guided educational research at The Open University for the 
past forty years. As this chapter and the previous one have shown, over that 
time great progress has been made towards that vision. Online courses at The 
Open University and other institutions have successfully taught hundreds, even 
thousands, of students on formal courses for over two decades. Over the last fif- 
teen years, the availability of open educational resources, and informal learning 
opportunities such as OpenLearn and iTunesU, have increased their scale and 
scope. Citizen science projects involving hundreds of thousands of people run 
on an international scale, providing opportunities to learn about the scientific 
method and to put it into practice to generate new knowledge. 

The arrival of MOOCs on the scene was part of this expansion of educa- 
tional opportunities, and researchers were immediately able to start making 
connections between this new format and previous decades of experience. 
The landscape of learning at scale continues to change. New providers emerge, 
their business models associated with new challenges and opportunities. By 
looking beyond the different formats and models to a vision of what can be 
achieved in the future, it is possible to identify and work towards objectives 
that make that vision achievable, focusing on teams, learning design, access, 
assessment, accreditation and, perhaps most important, the pedagogy that 
shapes these opportunities. 
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CHAPTER 4 


Accessible Inclusive 
Learning: Foundations 


Tim Coughlan, Kate Lister, Jane Seale, 
Eileen Scanlon and Martin Weller 


As a foundation to understanding how to be accessible and inclusive in TEL 
research, this chapter explores different conceptualisations of ‘openness’ and 
accessibility’. Using a range of examples, we then highlight how research pro- 
jects take a particular orientation towards inclusiveness through their goals, 
methods and platforms. Technical accessibility, and opening up the potential to 
access education, are essential to an inclusive approach, but alone they rarely 
provide the basis for equitable learning. The examples therefore provide par- 
ticular insights into how technological innovations need to be considered in 
concert with pedagogy. We then explore how our research has identified gaps 
and factors in digital inclusion for particular groups, and has been orientated 
towards designing for diverse audiences in response. An emphasis on processes 
and practices has emerged in both the accessibility and open education spaces, 
and we describe a practical example in which the OU has successfully embedded 
research-informed institutional practice through the Securing Greater Accessi- 
bility (SeGA) initiative. 
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The vision: Learning is accessible for everyone 


One of the most persistent themes in discussions around technology in educa- 
tion is the idea that technology can affect access to learning. This can be seen 
as positive or negative, and it is often more complex than it seems. If computers 
can convert the text in a web page into spoken word, or the spoken words on a 
video into captions, have we made the learning accessible to deaf or blind stu- 
dents? Most likely we have made an important step in the right direction, but 
this might be only one challenge in the wider pedagogy and student experience. 
If MOOCs can teach thousands for free without any cost or entry requirements, 
does that mean they are increasing access? Perhaps, but are they also creating 
barriers for some through the pedagogical and technical design? In this chapter 
we will unpack how these issues have been tackled through research. 


What do we mean by accessibility and openness? 


‘Accessible’ can mean different things to different people in different contexts. 
Similarly, when we say that something is ‘open, we have a broad sense of what 
this entails, but open to whom, when, and how? While it can be unhelpful to get 
bogged down in definitions, we should consider what these terms can mean. 
Hopefully this avoids some confusion that might otherwise arise, but it also 
gives us a starting point to think about what we are trying to achieve. 

Let’s begin with the model of open access education provided by The Open 
University (OU). This was developed to tackle the issue of supporting people to 
enter higher education who are traditionally excluded from it. What makes it 
‘open access’ is the removal of entry requirements and the flexibility provided 
by support for study at variable levels of intensity, part time, and at a distance. 
It has been remarkably successful, with the OU’s approach adapted in many 
institutions in countries around the globe. The model was enabled by technol- 
ogy and services from the very beginning. Radio, television, the postal service, 
printed materials, videos, DVDs, and the Internet have all been essential. 

This model also presents an ongoing challenge. As the aim is to be ‘open to 
all, and to provide opportunities for those otherwise excluded from education, 
the open access model had to include a focus on making learning accessible for 
people with disabilities, with the recognition that traditional higher education 
included barriers that might prevent them from studying. Being open to all 
created requirements for being accessible to an extent and scale that might not 
otherwise have been considered necessary, particularly in the past, when inclu- 
sion was not a major concern for most educational institutions. 

How does this compare to a different model of openness? Let’s consider 
Open Educational Resources (OER), including offshoots of this such as Open 
Textbooks. Here, openness is not just about access, but about the freedom for 
educators and for students to share and reuse or adapt resources to their needs, 
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Figure 4.1: Sharing your materials is an important part of OER - Image by 
Bryan Mathers reprinted under creative commons license. 


free of charge. OER are free to use as long as the licence conditions are not 
breached. This provides a legal definition of openness, not just an educational 
or social one. It is also notable that the use of OER are intended to open up 
opportunities for educators as much as it is intended directly for students, since 
the benefits may be harnessed by educators and then benefit both their practice 
and their students’ learning. 

A different, less-defined model of openness is found in MOOCs and similar 
forms of online open learning at scale. These tend to be free or have lower costs 
involved than other forms of post-secondary education, and like the OU, they 
avoid entry requirements. But unlike OER, there are often limitations on the 
rights of learners and educators to make use of the materials. Anyone can create 
and share OER, but MOOC platforms may not be open in the sense that they 
will only publish certain courses or work with particular institutions. They may 
argue that being closed in this way supports quality control, but this approach 
could also be seen as exclusive rather than inclusive. 

The focus of the MOOC agenda has been on producing platforms and content 
that can be accessed at large scale with low barriers. However, the need to keep 
costs low often means limited support for learners is available. In the OU, and in 
most traditional educational institutions, there are teachers and student-facing 
support staff to guide students and to adapt the learning to their needs. These 
forms of individual support do not generally exist within the MOOC approach. 

Another less well defined (but often discussed) use of ‘oper is that of Open 
Educational Practices. In this case, the individual or institution aim to reduce 
any boundaries surrounding them. Rather than teaching solely their own stu- 
dents within a VLE, an open practitioner could potentially teach through con- 
versing and sharing their work on a multitude of platforms, and by working 
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with others from around the globe. Open practices involve an open attitude 
towards mixing the role of institutions, resources, and platforms. 

Like openness, the term ‘accessible’ is used in several different ways. In this 
chapter we mainly focus on accessibility in relation to disablement. This pro- 
vides a specific focus to which persistent intent can be applied to make a differ- 
ence to learners. However, the notion of whether something is ‘accessible’ can 
be used to focus attention for other populations too. For example, whether the 
language used in a course is accessible to particular audiences, or is too com- 
plex such that it might make the learning inaccessible to them (Rets et al., 2019; 
Coughlan & Goff, 2019). 

As with ‘open; there are technological and legal influences working alongside 
ideology and theory. Most technical consideration of online accessibility centres 
on whether resources and platforms meet specific criteria laid out by the widely- 
used Web Content Accessibility Guidelines (WCAG). But accessibility research 
also explores how a particular technology or service can be designed to enable 
or exclude particular users. Accessibility and assistive technology research has 
tended to emerge from the computing disciplines and technology companies. 

An alternative perspective to this can be primarily pedagogical, asking 
whether specific learning activities or outcomes are taught in ways that exclude 
particular individuals. A focus on the aims for learning experiences and out- 
comes can then support exploration of how to achieve these outcomes in an 
accessible way. 

We can also take a broader perspective to ask what barriers are experienced 
when a person tries to access education, and who may be missing out. Research 
through this approach can be driven by reports or observations from learner 
perspectives, or by data that highlights the relative gaps in engagement or 
attainment with education by particular groups. 

Finally, we can conceive of accessibility as a quality achieved by the ways in 
which organisations, as combinations of people, systems, and processes, work 
together. This perspective recognises the holistic nature of support for acces- 
sible learning that cannot be reduced to a single technology or job role. 

In the rest of this chapter we will explore how examples of these conceptions, 
or combinations of them, has driven research and produced greater under- 
standing of what it means to make learning accessible to all. 


How can we make learning experiences available to all? 


In taking a pedagogical perspective on accessibility, we noted above that par- 
ticular types of learning activities create specific barriers. In this section we 
describe how a persistent intent on enabling access to STEM laboratory and 
field work has driven research over many years. In this, researchers have looked 
to harness the cutting edge technologies of the day and envision how these can 
become embedded in mainstream teaching and learning. 


Accessible Inclusive Learning: Foundations 55 


It has been argued that science, technology, engineering, and maths (STEM) 
subjects raise some very particular and stubborn challenges for access in the 
areas of practical work, such as laboratory and field activities (IOP 2017). The 
impact of inaccessible field or lab activities can also be exacerbated as many 
STEM qualifications are accredited by professional bodies, and these bodies 
often list practical work as a requirement for accreditation. However, the chal- 
lenges have attracted sustained attention and persistent intent to widen oppor- 
tunities (Pearson et al., 2019a). 

Traditionally, laboratory work requires students to be present in a lab to 
manipulate apparatus. The requirements to be in a particular location and to 
perform particular physical activities with apparatus can present accessibility 
challenges. In response, remote laboratories aim to provide manipulation or 
control of real apparatus through interfaces at a distance. Such approaches can 
expand access to important science learning experiences for students with dis- 
abilities, and for all students studying at a distance from laboratory facilities. 

The Practical Experimentation by Accessing Remote Laboratories (PEARL) 
project explored ways in which computers could be used to give high qual- 
ity learning experiences in science and engineering education by bringing the 
teaching laboratory to the students, giving flexibility in terms of time and loca- 
tion. The tools and activities created in the project were also designed to be 
accessible to disabled students using assistive technology, such as screen readers. 

A model of collaborative working underpinned the learning activities, with 
students working with peers and receiving comments from tutors. The complex 
system which was developed provided a structure which combined tools for 
collaboration with technology to control the equipment, network server and 
interface technologies, and streaming media, video cameras, and microphones 
to provide the means of observation and communication. 

As one of a number of explorations in different institutions, the project 
involved a re-versioning of an introductory Open University science experi- 
ment usually performed by students co-located at a residential school, to allow 
remote operation of a spectrometer to measure wavelengths. Scanlon et al., 
(2004) describe the interface through which students working at a distance 
could take part in this experiment. Evaluations with disabled students con- 
firmed that they could use the interface effectively. While the remote approach 
was found to be a different experience, sufficient equivalence with the original 
laboratory experiment could be achieved. 

Students and academics were supportive of the PEARL approach, which has 
continued to develop. Cooper and Ferriera (2009) summarized the lessons 
learned about the design and implementation of remote laboratories based on 
these experiences, stressing the importance of having a well-defined pedagogic 
strategy, of removing accessibility barriers, and the need for ease of automation 
and remote control. 

Deployment of these ideas at scale then became the focus. In 2013 the 
OU, with support from the Wolfson Foundation, launched the OpenScience 
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Laboratory. This virtual lab allows students to carry out experiments online, 
bringing interactive practical science to students anywhere and anytime they 
have Internet access. As with PEARL, the aim is to provide access to real physi- 
cal instruments and equipment through robotically controlled experiments, 
but the laboratory platform also provides a basis for interactive screen experi- 
ments; virtual instruments and labs; immersive 3D experiments; virtual field 
trips; and mass participation ‘citizen science’ networks (Garrow et al., 2013; 
Villasclaras- Fernandez et al., 2013). 

This initiative led to the development of the OpenSTEM laboratories, a suite 
of distinct labs incorporating the OpenScience Lab, the OpenScience Obser- 
vatories and the OpenEngineering Lab. The OpenScience Observatories pro- 
vide access to two remotely operated optical telescopes based in Tenerife, and a 
radio telescope based at the OU campus in Milton Keynes. The OpenEngineer- 
ing Lab allows practical lab-based teaching at a distance covering engineering, 
electronics, control, materials and robotics. Together these connect students to 
instrumentation, data and equipment for practical enquiries over the Internet, 
where time and distance is no longer a barrier. 

In these developments it is important to maintain consideration of the physical 
and social aspects of the laboratory activities. Experiences that connect students 
to the on-campus labs allow students to acquire and practise lab-based skills. Lab 
casts provide an interactive experience by connecting students and lecturers via 
web streaming in a way that provides a live social experience at a distance. 

Another activity that can often be inaccessible to learners, but is a recognised 
component of science and many other subjects is fieldwork. In subjects such as 
geology or biosciences, study in the field is seen as essential and is known to 
support conceptual and practical understanding (Elkins & Elkins, 2007; Scott 
et al., 2012). The terrain and location of many field sites of interest present bar- 
riers to those with mobility challenges. A persistent intent through research 
spanning more than a decade has led to greater understanding of how remote 
access to field work can be achieved. 

The Enabling Remote Activity (ERA) approach was first prompted as a 
response to an enquiry from staff and a student who was using a wheelchair 
and studying earth sciences. They highlighted the possibility of using audio 
and video to communicate with students unable to reach a particular field site. 
From this, the wider issue of remote access to fieldwork was tackled through 
the development of a flexible toolkit (Collins et al. 2016). 

The right field site may be expensive and time consuming to reach, so issues of 
access arise not only for students with mobility challenges. Cost and availability 
are often prohibitive factors that exclude students from access to field experiences. 

As with remote laboratories, technologies including networks, sensors and 
cameras offer the potential to create remote presence and interaction with a 
field site. However in fieldwork, variability in locations and the need for mobil- 
ity in order to explore the field site create further challenges. Technical solu- 
tions to these have pushed at the boundaries of what can be achieved with 
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Figure 4.2: Enabling Remote Activity (ERA) field trials. Copyright Mark Gaved, 
The Open University. 


mobile networking and portable technologies. Meanwhile, the practical and 
pedagogical approaches employed in remote fieldwork have been a particularly 
interesting focus for research. 

Findings from ERA include the value of multiple communications chan- 
nels, with voice providing an important direct link between individuals, video 
providing a sense of presence and live interaction, and photographs important 
for details. A combination of these are used according to the learning activ- 
ity. Rapid deployment of equipment was also key in order to fit in with field 
trip schedules and avoid students being left behind. The ERA approach has 
focused on getting people as close to the field site as possible, such that they can 
gain from as much of the field experience as possible and only use technology 
to overcome the parts that are inaccessible. This attends to the argument that 
social and experiential aspects of fieldwork are important and could be lost, 
resulting in a further form of inequity (Collins et al., 2016). Further develop- 
ment of the ERA approach has led to the development of an accessible field trip 
in Connemara (part of the National Science Foundation funded Geopath Extra 
project, 2017) and an inclusive field course in Anglesey (part of the Office for 
Students funded IncSTEM project). 

The value of a persistent intent in this area has been that we have developed 
and tested multiple designs for technology-enhanced learning activities. So 
where the value of having learners in close proximity to the field site was rec- 
ognised in ERA, this could itself become a barriers as it might not always be 
possible to have the learners close by. As such, alternative designs for social and 
collaborative field activities were also explored. The ‘Out There and In Here’ 
(OTIH) project took the ERA findings in a different direction. It explored how 
to set up a ‘command centre’ in a classroom setting, where a group of remote 
students could learn through dialogue and collaboration with their field- 
based peers. Trials and evaluations looked to find ways to design for balance, 
such that all students involved could have an equitable and valuable learning 
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Figure 4.3: Classroom (top) and field-based students in an ‘Out There and 
In Here’ remote fieldwork trial. Copyright The Open University. 


experience (Coughlan et al., 2010). The outcomes showed that there are differ- 
ent strengths to each situation - a field-based student can capture data, but a 
classroom setting may be better suited to analysing or identifying it. There are 
also different challenges to each experience - the field-based student or teacher 
can feel pressured to provide material to their remote peers, who require this 
for the experience to be effective (Coughlan et al., 2011). 

These examples of designing and evaluating remote laboratory and remote 
fieldwork experiences show how research can utilise technology to enable 
access to specific activities that are commonly inaccessible. While these expe- 
riences are not an exact replication, they can be designed to offer learning 
outcomes that would otherwise be lost. They can also prompt thinking about 
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new pedagogical approaches and help to unpick what the aimed-for learning 
is, which may be otherwise implicit or taken for granted. Research in this area 
used cutting edge technologies and overcame a lack of suitable technologies 
to explore potential solutions in advance of these approaches reaching main- 
stream use. At the same time, it requires an awareness of the social and expe- 
riential aspects of learning to really evaluate how equitable, accessible learning 
can be achieved. 


Broadening our understanding of accessibility 
from availability to equity 


The multiple conceptions of openness and accessibility, and the examples of 
research that aims to make field and lab work accessible, both show that mak- 
ing learning accessible is not a simple endeavour. For example, the PEARL eval- 
uation raises the notion of whether an accessible remote experience is similar 
or equivalent to that in the lab, and the ERA and OTIH projects highlight the 
importance of less formal aspects, such as the shared social experience of being 
on a field trip, in a particular place and in the company of fellow students. 

These issues can impact on the learner but would not be captured by a nar- 
row definition of accessibility. In this section we delve further into this sense 
that making learning available is not sufficient. The Beyond Prototypes frame- 
work highlights the importance of evaluation and evidence to drive TEL inno- 
vation. In this area, there is a wealth of evidence of persistent gaps in access, 
attainment, and experience of learning at scale. A review of this suggests that 
a simple notion of educational access (i.e. that a person was technically able to 
join a course of study) results in significantly different outcomes for learners 
with different characteristics. We then consider how it is possible to respond by 
understanding the needs of particular groups, in order to facilitate their equi- 
table access to education. 

Analysing data on registration, completion, and attainment in post-secondary 
study, the picture that emerges challenges simplistic visions of making learning 
accessible or open to all. Richardson has conducted a number of analyses in 
this area looking at specific groups such as categories of disability or ethnic- 
ity. The findings with regards to disabilities show a complex picture, including 
that students with declared autistic spectrum disorders studying at The Open 
University were just as likely to complete, pass, and obtain good grades in their 
modules as students without any declared disabilities (Richardson, 2017), and 
that students who are deaf or hard of hearing were more likely to complete their 
modules than their non-disabled peers (Richardson, 2015a). While students 
with dyslexia or other specific learning difficulties were just as likely to com- 
plete their module as students without declared disabilities, they were less likely 
to pass or to obtain good grades (Richardson, 2015b). In contrast, students with 
declared mental health difficulties, or with visual impairments, were less likely 
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to complete their modules, and less likely to pass them (Richardson, 2015c; 
Richardson, 2015d). 

Richardson's work highlights that considering all people with declared disa- 
bilities together as a meaningful group is problematic, since it hides differences 
that become visible when we look at particular sub-groups of ‘disabled students. 
The types of barriers to equitable learning faced by students with dyslexia, and 
the technologies and support actions that would enable equitable learning for 
them, are not the same as those faced by a person with mental health difficul- 
ties. At the same time, these analyses also highlight the importance of account- 
ing for intersectionality, where multiple characteristics of a person may impact 
on the accessibility of learning. Richardson finds substantial proportions of stu- 
dents declare multiple disabilities, and that these groups tend to be less likely to 
succeed. Focusing on disability in isolation could be problematic, since other 
factors such as prior educational qualifications and ethnicity can also be shown 
to correlate with student success. 

With regards to ethnicity, persistent and ubiquitous gaps in attainment for 
ethnic minority students have been identified in UK higher education when 
compared to white students. A particularly interesting finding from the perspec- 
tive of making learning accessible is that these gaps are only partly explained 
by entry qualifications. One analysis extrapolates that around half of the attain- 
ment gap in higher education can be explained by poor attainment in earlier 
stages of education, but that the other half cannot be explained by this measure 
of academic ability. This may be occurring due to unknown factors within the 
higher educational experience, which could include discrimination or more 
subtle processes through which these students are not supported to perform 
(Richardson, 2015e). We do not fully understand where and why these gaps 
occur, but this work suggests that a simple notion of accessibility in education 
- that a person can manage to register, engage, and complete a course - does 
not necessarily lead to equitable educational outcomes. We need to consider 
the experience as a whole and identify elements of teaching that a person might 
find inaccessible or which might lead to inequality that impacts on outcomes. 

Richardson's findings draw on data about students taking part in formal open 
access education at The Open University. But what about OER and MOOCs? 
Arguments have been made that these approaches could lead to greater inclu- 
sion in higher education by lowering barriers of cost and flexibility (Lane, 
2008). However, as Farrow et al. (2015) report, non-formal users of OER are 
likely to already hold a degree, or to be currently studying on a formal higher 
education course. This is not to say that OER are not supporting some widen- 
ing of access, but it suggests that they may be primarily useful to those who are 
already benefiting from formal study. MOOC platforms have been found to 
have substantial failings with regards to accessibility for disabled learners, and 
those involved in the production and presentation of MOOCs are still develop- 
ing strategies to provide disabled learners with a good study experience (Iniesto 
et al., 2016). OER are often derived from existing formal course materials and 
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so can replicate some of the barriers for non-traditional audiences that exist in 
these, while removing the active support and encouragement of learners that 
would perhaps support them to succeed in a formal educational setting (Lane, 
2008; Coughlan & Goff, 2019). 

Richardson (2015e) notes that we can struggle to identify the causes of ineq- 
uity for particular groups, even where we can see the results in quantitative 
analyses. A range of studies have provided richer insights into ways in which 
educational provision can be problematic for particular groups, or can be 
designed with recognition of these problems. Having established these issues, 
how can we move from a notion of accessible or open as equating to ‘available, 
to something more equitable? What might we need to understand in order to 
design to close these gaps? The next section addresses these issues. 


Responding to the diversity of contexts and individuals 


We have already introduced some examples where new uses of technology cre- 
ate a basis for educational opportunities and increased access. In this section 
we will focus further on ways of designing for audiences and contexts. 

An important point to start with is that while the focus of our attention is 
often rightly on pedagogy, there may be practical and social issues that impede 
access to education and which need to be understood and adapted to any spe- 
cific context. For example, working to design educational technology solutions 
for the context of refugee camps, Alain et al. (2018) argue that issues such as 
prior and current disruption to formal education, language barriers, and the 
availability of teachers, need to be considered if technology-based interventions 
are to effectively engage children. Each refugee camp will have different social, 
physical, and technological resources and limitations that can be employed to 
create informal educational opportunities in these settings, and further distinct 
challenges are faced in situations where refugee children are to be integrated 
into local school systems. However there has been a tendency for initiatives 
around refugee education to design for scale in an homogeneous way, with a 
lack of awareness or potential to adapt to these contextual differences. 

The move towards greater online and hybrid learning has enormous potential 
for making access to education easier for many populations. However, when 
this results in the removal of other means of study it can create new forms of 
exclusion. A prime example of this is in education for students in secure envi- 
ronments. A study of universities across four different countries highlighted 
that prisoners found it increasingly difficult to access distance education, with 
risk-averse correctional systems prohibiting or restricting access to the Internet 
and to computers (Farley et al., 2016). Solutions can be used that present an 
offline digital version of materials, or printed versions can be provided. How- 
ever, it is important that these solutions are designed in such a way that they 
provide the intended learning experience and do not become an afterthought. 
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More subtle barriers to online learning are present for people who lack skills 
or confidence in ICT. Concepts related to digital inclusion or exclusion have 
been described and debated to highlight the increasing reliance on digital tech- 
nologies across society and the impacts this can have. While it was appealing to 
think that younger generations of ‘digital natives’ had different expectations and 
skills with technology when compared to older ‘digital immigrants, research by 
Jones and Shao (2011) refuted such a simple age-based dichotomy. They also 
found that many of the new technologies discussed in educational research, 
such as virtual reality, wikis, or blogs, were not ones that students made use 
of or expected to be used. At the same time, there were clear signs of the ris- 
ing general use of social media, mobile technologies, and online multimedia. 
Where these become key to learning - perhaps as part of open educational 
practices — there are further possibilities to include or exclude. 

It is therefore important to understand and critique the factors that medi- 
ate the relationships between digital technologies and learners. Seale (2014) 
argued that it is necessary to look beyond a simplistic notion of accessi- 
bility in order to understand the factors impacting on disabled students. If 
we focus only on whether technologies are available and if the person can 
access them, there is a risk that the complexity of the relationship that disa- 
bled students have with their technologies and their educational institutions 
will be ignored. We need to avoid a situation where we only consider the 
relationship between student and institution as one of receiver and provider 
of resources. 

How then can we think more broadly about accessibility in terms of the rela- 
tionships between students, resources, and educational institutions? Drawing 
on the ideas of digital inclusion researchers such as Eynon (2009) and van Dijk 
(2005) who talk about the resources that people need in order to be citizens of a 
digital society, Seale (2014) identified a range of factors that potentially mediate 
the relationship between students and their institutions: 


Temporal Resources: The time available to disabled students to invest 
in learning how to use new technologies. Time can be limited and 
insufficient due to the additional study burden that disabled students 
experience — particularly if their courses have not adopted an inclusive 
approach to teaching or made reasonable adjustments. 

Mental Resources: The knowledge, awareness and skills that disabled 
students possess that means they are confident and competent in using 
a wide range of technologies and have created a wide range of strategies 
for using their technologies to support their learning. 

Social Resources: The range of formal and informal support networks 
such as academic peers, tutors, friends and family that disabled students 
can draw on. 

Cultural Resources: A climate or environment where disabled stu- 
dents are perceived as legitimate technology users, where there is an 
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expectation that they (along with everyone else in the community) can 
and should be using technology. 

Material Resources: Access to a range of generic and assistive technol- 
ogies, some of which disabled students may personally own, some of 
which are provided by the institution. 


In this case, the access to technology (material resources) is just one aspect 
of a broader view. Even in considering these material resources, we should 
note that only some of what a disabled student might use to learn is provided 
by the institution. 

Building on this framework, Seale (2013) and Seale et al. (2015) apply a 
‘Digital Capital’ framework to the understanding of the relationship between 
disabled students, their technologies and the institutions in which they study. 
Drawing on the ideas of earlier research (Bourdieu, 1997; Putnam, 2000, 
Selwyn, 2004) two key concepts were proposed: ‘Digital Cultural Capital’ and 
‘Digital Social Capital’ 

The acquisition of digital cultural capital is exemplified by individuals invest- 
ing time in improving their technology knowledge and competencies through 
informal or formal learning opportunities, as well as a socialization into tech- 
nology use and ‘techno-culture’ through family, peers and media. 

Digital social capital is developed through, for example, the networks of 
‘technological contacts’ and support that people have, which can be face to face 
(e.g. family, friends, tutors) or remote (e.g. online help facilities). 

Seale (2013) used this digital capital framework to analyse data collected from 
30 disabled students regarding their experiences of using technology to support 
their learning. Results indicated that disabled students possessed a significant 
amount of digital cultural capital and a fair amount of digital social capital. Seale 
observed however that for some disabled students, this cultural and social digi- 
tal capital did not appear strong enough. For example, some disabled students 
appeared to be affected by the extent to which using specialist technologies 
marked them out as different. Seale et al. (2015) also applied the digital capital 
framework to analyse the experiences of 175 students with declared disabilities 
regarding their use of technology to support their learning. Results suggested 
that while these students do have access to social and cultural resources; some- 
times these resources are not appropriate or effective (e.g. school-based ICT 
qualifications) or disabled students are not drawing on all the possible resources 
available to them (e.g. non-institutional based support or support from disabled 
students). This means that disabled students can lack the ‘right’ kind of digital 
capital to enable them to succeed within higher education environments. 

Using an analytical framework that goes beyond a simple conception of 
accessibility, this research suggests that higher education institutions may need 
to conceptualise and organise technology related support services for disa- 
bled students differently. There is a need to think beyond simplistic notions of 
access, availability and skills training. 
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To summarise, for learning to be accessible to all means more than for it to 
just be available, because inequitable situations arise through a lack of consid- 
eration of how contexts and individuals are related to the resources and support 
for learning. Analyses of student data can highlight inequalities of engagement 
or attainment, and qualitative studies offer understanding of the challenges 
faced in particular contexts. To build on this, we look in the next section at 
procedural and practice-based approaches to understanding and achieving 
accessible learning. 


Consider process and practice, not just artefacts and outcomes 


In research and scholarship around openness and accessibility, there has been 
a growing recognition of the need to consider process and practice, rather than 
only artefacts and outcomes. This makes sense to educational technology in 
particular for a variety of reasons. For example, if we think in terms of process 
and practice it is easier to explore how we could adapt technological artefacts 
to be better suited to particular contexts, and support individuals to be aware 
and able to use them. If we consider the experience of a student as a process that 
includes multiple events that could have short and long term impacts on their 
learning and attainment over time, we are better placed to identify why gaps in 
attainment might appear. 

The importance of taking a process view of accessibility has emerged more 
recently with the suggestion that any artefact can only be considered acces- 
sible in relation to a particular person trying to use it; or as Cooper et al. 
(2012) put it: “The focus of WCAG (Web Content Accessibility Guidelines) 
is on the technical artefact — i.e. the “web page’, not on users and user goals” 
(pg.1), yet “accessibility is a property of the relation between the user and 
the resource in the context of how that is mediated; not a property of the 
resource” (pg. 2). While WCAG is very important in defining characteristics 
of a resource that should support it to be accessible, it is not sufficient to 
ensure equitable experiences because it gives no sense of how it is used in 
practice by particular audiences and (for example) the mental, social, tem- 
poral and cultural resources that Seale (2013, 2015) explores in her analyses 
of disabled student experiences. 

There is a related trend in open education research. Much effort has been 
focused on tasks such as defining what an open educational resource should be 
in terms of legal or technical infrastructure, and in devising implementations 
of this such as Open Textbooks. But until recently, it has been less common to 
conduct research to understand how people do, or could, engage in practices 
around using these resources (Weller et al., 2015). 

In both cases, the original focus on artefacts could be attributed to a desire 
to build a broadly applicable basis for change at scale - the wide use of WCAG 
standards to evaluate web page accessibility, and of Creative Commons licences 
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for sharing of OER are clearly influential developments that have achieved 
widespread impact. But the move towards practice and process is prompted by 
the understanding raised in the previous section, that truly providing access 
means responding to individual contexts and needs that are subject to change 
over time. 

A focus on practices in open education helps us to understand the benefits that 
stem from sharing resources under open licences, and the barriers that could 
prevent these from being realised. For example, when Pitt (2015) explored edu- 
cator perceptions on the impact of introducing OpenStax College open text- 
books, they found a range of responses. A key driver for openly-licensed texts 
was the need to serve students for whom the expense of proprietary textbooks 
was a barrier, and to save money for cash-strapped institutions. Cost savings 
were certainly a theme in responses, but when asked what the main impact of 
introducing the textbooks was on their teaching practice, the most common 
responses were that it had made teaching easier (29%), or had led to innova- 
tions in their teaching practice (25%). These educators reported that they could 
build on the text, adapt it to suit their classes and students, feel more able to 
combine it with other resources, and be creative in a way that a closed propri- 
etary text would not support. One stated that with open licences that support 
adaptation of content “the book is my servant, I am not its servant” (pg. 148). 

At the same time, reusing and adapting OER can be challenging. Educa- 
tional resources are produced for a particular context and audience, and it can 
be problematic to reuse these with other audiences. Research in this area has 
analysed processes of reuse to highlight how participation in remixing can 
improve the relevance of resources to particular audiences, while maintain- 
ing the original objectives of the material. In the ‘Bridge to Success’ project, 
courses designed by The Open University as an introduction to study for those 
with limited prior educational experiences, were released as OER and remixed 
for use with US audiences of underserved learners. The collaborative approach 
engaged US-based authors in adapting the content to the needs of their own 
students. Changes were made at various levels including to find appropriate 
language, or to increase content in areas that were more important for the 
intended US pre-college audience, when compared to the original UK audi- 
ence. These included, for example, fractions in mathematics. In addition, pro- 
motional activity to introduce the courses, and to work with educators to find 
ways in which it would be best integrated into their teaching, was essential to 
gaining uptake across a range of settings. Although intended to be useful to col- 
leges, the resources were also used in unexpected contexts, such as charitable 
organisations who worked with underserved groups (Coughlan et al., 2013; 
Coughlan et al., 2019). 

Given that open online courses are not necessarily reaching underserved 
audiences, related practices of targeted collaboration in the creation and use 
of OER have been applied in other projects. When creating courses with the 
intent of reaching a particular underserved audience, our research identified 
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ways in which collaboration with a ‘Learner Representative Partner’ could be 
beneficial. The research analysed the processes of creating a suite of six open 
online courses in collaboration with different organisations, and each aimed at 
encouraging engagement with higher education from an identified target audi- 
ence, for example those working in healthcare assistant roles, or looking to start 
their own business (Coughlan & Goff, 2019). The partners informed the design 
of the courses, including the reuse of their own resources and media, and in 
deciding on appropriate language and content for the audience. The partners 
facilitated the courses to embody student-centred strategies, influenced the 
language used in the course materials, provided authentic case studies from 
people similar to the audience, and acted to highlight areas of ‘decoupling’ 
where academic practices did not fit with the desire to widen participation. 

An interesting example of research that recognises the need to consider 
process, practice, and particularly people, is the EU4All (European Unified 
Approach for Assisted Life-long Learning) project, which took place between 
2006 and 2011. The project involved 13 European partners and aimed to 
research and develop ways to make life-long learning at higher education level 
accessible to disabled people. This involved wide stakeholder engagement, 
individualised design approaches and extensive evaluation; it resulted in the 
creation of a model of ‘professionalism in accessibility’ (McAndrew et al., 2012) 
and a learner-centred framework for personalisation of content and service. 
Other, less tangible outputs from the project include key lessons learned about 
operationalising accessibility; it became clear in the EU4All project that acces- 
sibility cannot be achieved in an educational institution if it is viewed only as a 
technical consideration. Understanding user needs, experience and preferences 
is critical if accessibility is to be embedded. As accessibility frequently means 
different things to different stakeholders, human engagement through a multi- 
faceted, multi-stakeholder approach is an essential part of this process. 

The EU4All project drew on work by Seale (2006) that explored adopting 
a holistic view of the stakeholders and activities involved in achieving acces- 
sible learning. Seale had investigated the perspectives of different stakehold- 
ers, including their issues and concerns, in an attempt to amplify these diverse 
voices and provide a cross-sectional view of accessibility. Drawing on personal 
experience, Seale stresses that for accessibility to be realised, a range of stake- 
holders need not only to be involved but to actively form strategic partnerships, 
and that this "cannot happen successfully unless each stakeholder understands 
the different perspectives of each of the other stakeholders” (2006, pg. 4). The 
EU4AIl researchers modelled their work on Seale’s, ensuring all Seale’s identi- 
fied stakeholders had a voice in the EU4AIl project, as well as adding other 
stakeholder groups. The resulting ‘model of professionalism in accessibility’ 
placed strong emphasis on a ‘holistic view of accessibility’ with stakeholder 
engagement and human beings ‘in the loop’ (McAndrew et al, 2012). 

These lessons became part of Open University practices through the devel- 
opment of the SeGA (Securing Greater Accessibility) initiative. In 2010 it was 
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becoming clear in The Open University that there was a disconnect between 
student support teams and academic staff, which was resulting in academics 
not being aware, when designing learning content, of feedback about what stu- 
dents were finding inaccessible. This frequently resulted in costly, retrofitted 
reasonable adjustments being made by student support units that could have 
been avoided through inclusive design. Pockets of good practice did exist, but 
responsibility for supporting disabled students was dispersed across a number 
of academic and non-academic units and information was difficult to find; the 
good practice that existed was not systematised and was ‘decoupled’ (Meyer 
and Rowan, 1977) from institutional strategy, policy and business as usual. 

Acknowledging these issues, several stakeholders in different areas joined 
forces to lobby for a change; an initiative that would begin to address these 
issues and operationalise accessibility in a way that was systematic, consist- 
ent and sustainable. This would be a whole-institution and whole-product and 
service life-cycle approach, with stakeholders from both academic and support 
units. The SeGA initiative was launched with the objectives of: 


e Clarification of responsibility and accountability for leading on and deliver- 
ing accessibility. 

e Improved access to the curriculum for disabled students. 

e Improved understanding of staff roles and responsibilities regarding 
accessibility. 

e Improved documentation of how the reasonable adjustments offered to stu- 
dents have been arrived at. 

e Reduced overall cost for providing adjustments to disabled students. 

e Improved organisational knowledge of enabling accessibility best practice. 

e Improved visibility of the levels of accessibility afforded to students within 
courses and programmes. 


(Cooper, 2014) 

In its conception, SeGA drew heavily on Seale (2006) and the EU4All findings 
and resulting model of professionalism in accessibility (McAndrew et al, 2012). It 
also incorporated aspects of Communities of Practice theory (Lave and Wenger, 
1991) and the social model of disability (Oliver, 1983). Initially it worked to 
clarify responsibilities, processes and systems, and, through a network of acces- 
sibility coordinators and champions, to ensure information was easily available 
to people who needed it. It acts to bring different stakeholders together and to 
empower the voice of students and other stakeholders in operationalising acces- 
sibility. In short, to be the ‘humans in the loop’ (McAndrew et al, 2012). 

SeGA is embedded in the processes of teaching through ‘Accessibility Coor- 
dinators, members of staff who take responsibility to advocate for accessibility 
in the teaching in their faculty and school areas (Pearson et al., 2019b). SeGA 
supports the Accessibility Coordinators with regular training events, and coor- 
dinates a working group through which Accessibility Coordinators can share 
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their current practices and challenges, and take part in projects to improve the 
accessibility of learning and teaching at an institutional level. Recent projects 
have included creating guidance for external assessors and critical readers of 
module material, and embedding this guidance in the Curriculum Manage- 
ment System; changing university systems so that all undergraduate modules 
are required to have an ‘accessibility statement’ available to prospective stu- 
dents on the university website (as well as coordinating the writing of these 
statements); and refining the reasonable adjustment process so that examples 
of reasonable adjustments are recorded in a way that makes it easier to apply 
similar adjustments for future students. 

As the project became business as usual, SeGA expanded its reach beyond 
the curriculum to include other aspects of learning provision, for example it 
now includes representation from staff responsible for the production of OER, 
staff involved in the development and production of online learning tools and 
resources, and staff from areas such as Careers, the Library, Marketing and IT. 
A coordination group brings together representatives from across the univer- 
sity to identify and discuss key areas for attention, and a referrals panel brings 
together expertise to inform decision making on complex individual cases and 
course-level decisions that could impact on accessibility for students. 

SeGA also increasingly plays a conduit role between research and prac- 
tice, working to implement research findings into practice through training 
and staff networks, and ensuring current concerns and issues are shared with 
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research group channels to inform investigation. This acts to facilitate and 
empower ongoing persistent intent to make learning opportunities accessible 
to disabled learners. 


Conclusions 


This chapter started by considering that ‘accessible’ and ‘open’ are terms that 
can be defined and interpreted in different ways. They are often conceived of 
through technical definitions of licenses and web accessibility standards, or 
more broadly, that something is available and free or low in cost. However, 
making educational activities available is only a starting point to producing 
equity. Persistent intent to utilise TEL to widen access to field and lab-based 
activities has produced solutions which have been reused and adapted in teach- 
ing practice. At the same time, many of the interesting findings from these stud- 
ies were pedagogical in nature. These technologies provided a positive learning 
experience where previously there was none available, but this needed to be 
approached with a sensitivity to social elements of the experience, roles, and 
the nature of different environments. 

The Beyond Prototypes framework emphasises that innovations should be 
driven by evaluation and evidence. By compiling detailed understanding of 
uptake, completion and attainment by students with different characteristics, 
we can see that different issues exist for these groups in the Open University’s 
open access model. Inequity is also found in the wider sector, and in the uptake 
of OER and MOOCs. 

These quantitative findings provide a starting point to motivate and target 
research on inclusion, and lead to participatory and qualitative approaches to 
develop richer findings that address the ‘why’ behind these gaps, and to develop 
effective and appropriate interventions with audiences. In the past, it has not 
always been clear to TEL researchers that they should be cognisant of issues 
such as learner confidence or organisational risk aversion. However, these have 
come to the fore as factors that can create inequity with TEL. 

There are difficulties in moving between TEL research and mainstream 
practice, and the Beyond Prototypes framework encapsulates the complex- 
ity of this. However, there have been encouraging results in moving research 
and innovation on inclusion into organisational practice. A shift of focus in 
research beyond creating assistive technologies, and assessing the accessibility 
of artefacts, towards conceptualising accessibility as process that must include 
a range of stakeholders, informed the development of the Securing Greater 
Accessibility (SeGA) initiative which then became embedded in business as 
usual practices. Similarly, creating and using OER to widen participation have 
moved from a niche innovation to an established practice at the OU. Open- 
Learn now provides learning opportunities to millions of people every year, and 
Chapter 2 of this book provides further examples of practice-based projects to 
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create inclusion through open learning at scale. In the next chapter, we explore 
how these foundations are expanding to harness new ideas and technologies. 
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CHAPTER 5 


Accessible Inclusive Learning: Futures 


Tim Coughlan, Kate Lister, Jane Seale, 
Eileen Scanlon and Martin Weller 


The previous chapter on Accessible Inclusive Learning: Foundations outlined 
some key approaches and challenges when conducting research that seeks to make 
learning accessible to all. Here, we explore newer trends that are directing our cur- 
rent research and practice in this area. These promising directions include devis- 
ing models for global networks, the potential to collect and use data to understand 
learning experiences in new ways, and new opportunities arising through artifi- 
cial intelligence. By exploring current and recent projects around these areas, we 
also highlight some emerging tensions. Finally, we return to thinking about how 
we conduct research, considering how concepts of bricolage and guerrilla research 
are important in our methodological palette. 


The trajectory of the vision: Learning is accessible for everyone 


At The Open University (OU), we aim to be open to all in our ethos for teach- 
ing, and we look for similar approaches in our research. This means seeking to 
engage groups who are currently underserved in education, such as refugees 
or people from low socio-economic backgrounds who may not traditionally 
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access higher education. It also means we want to be open to engaging with 
people who have expertise and knowledge to offer, regardless of where or who 
they are. It means sharing and discussing our work through networks in ways 
that allow others to easily engage with it, or build on it (Weller, 2011). 

We begin by considering how institutional practices and research can be 
enhanced or driven by global and local collaboration. We follow this by explor- 
ing how new approaches to data gathering and analysis are required to realise 
the practice and process based views of accessibility described in the previous 
chapter. We then explore how research can broaden its audience, and broaden 
its impact, by moving from particular audiences and bricolage towards main- 
stream use. 


Working together through global and local networks 


We have argued in the previous chapter that understanding and adapting to 
the specific contexts of individuals, and of particular populations, is essential 
to create accessible education. At the same time, the advantages of working 
together around the world, and creating an impact at scale through mutual 
interest and discussion, are particularly visible in this area. 

Global collaboration has led to the Web Content Accessibility Guidelines 
(WCAG) that provide a common standard and criteria (W3C, 2018). The 
widespread application of this in education and elsewhere provides a means 
to promote and define expectations of accessibility that any website should be 
able to achieve. 

The development of Open Educational Resources (OER) also benefited from 
collaboration from stakeholders around the world, (e.g. Cape Town Open Edu- 
cation Declaration, undated). It has provided a well-defined means for anyone 
to share educational content and courses which supports other educators or 
students to reuse and adapt these (Creative Commons, 2016). OER provides 
a great example of the power of working together over time in a loosely-cou- 
pled way. This means that institutions, projects and individuals have declared 
enough consensus on their aims, and developed and adopted shared principles 
and models, such as the use of Creative Commons licencing. 

Because anyone can become involved in open education, and the collabora- 
tions can be loose or sporadic, it can be hard to understand what people are 
doing and the impact it is having. The OER World Map project tackles this 
issue and offers an example of working differently in the open, alongside 
insights into the way in which such a service can be designed to garner and 
sustain interest. The platform is designed to collect and visualise data on actors 
and activities in the open education space, providing a means to understand 
what initiatives, people and resources are available to engage with. While global 
in scope, it does not ignore local requirements. Tailored, country-specific maps 
can be produced, which provide insight that might be of specific interest to 
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practitioners in a particular region. There is functionality to display ‘Country 
Champions who are members engaged with the project in a particular region. 
The project combines a bottom-up approach to engage end users to contribute, 
alongside work to develop worldwide partnerships and strategic alignment to 
priorities. As OER becomes a mainstream approach for education, the avail- 
ability of information about resources and actors will become even more valu- 
able (Neumann & Farrow, 2018). 

In some contrast to the OER World Map, the Global OER Graduate Net- 
work (GO-GN) is focused on supporting the development of individual PhD 
researchers in a global context. Doctoral candidates are joined by experts, men- 
tors, and interested parties to form a community of practice. The network uses 
online webinars and face-to-face meetings to raise the profile of research in 
open education, offer support to students conducting research in the area, and 
to develop openness as a process of research. Because of the relative novelty 
of OER research, expertise and support for a doctoral researcher at their own 
institution may be limited, and connecting with other students and experts 
offers potential for greater impact. Furthermore, the network can provide a 
community where openness in the process of research is valued (de los Arcos 
et al., 2016). 

Networks such as GO-GN are not designed to be the exclusive ‘home’ of a 
researcher. People involved in GO-GN often belong to other networks and 
act as a bridge between these and GO-GN. For example, GO-GN has been an 
important further network for students who form part of the Leverhulme Trust 
funded Open World Learning (OWL) initiative. This initiative was devised to 
bring together diverse perspectives, with doctoral researchers coming to study 
at The Open University from all around the world (Institute of Educational Tech- 
nology, 2018). Many of these projects explore inclusion in OER and MOOCs, 
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including how MOOC learning varies by geo-cultural and socio-economic fac- 
tors, with differences identified in how learners behave based on location and 
prior education (Ritzi et al., 2019); and how OER presented in English may not 
be suitable for those for whom English is a second language, with the potential 
for language simplification to tackle this (Rets et al., 2019). 

A further model of global collaboration is found in the Ed-ICT network. This 
explores the role that ICTs can play in creating or removing disadvantages for 
students with disabilities in post-compulsory education. The approach taken 
here has been for themed workshops to be hosted in five different countries 
(Canada, Germany, Israel, UK, and USA). A core team formed from each coun- 
try attends each event to create coherent understanding across the workshops, 
but local practitioners, researchers, and students play an essential role in each 
workshop by sharing their perspectives and developing ideas and knowledge 
that are grounded in the local context. This is a wider instantiation of the ethos 
argued for by the network partners in their own work, which highlights the 
importance of student voice in research and technology development (e.g. 
Fitchen et al., 2014; Seale et al., 2013). 

The network brings to attention similarities and differences between the ways 
in which different countries approach accessibility and the factors that influ- 
ence these, such as government and institutional responsibilities, or common 
models of practice. Links can be drawn between prior research conducted in 
different countries and student populations, alongside practice-based issues. 

One example of this, which became the focus of an Ed-ICT workshop, is 
transitions. This can encompass situations such as the transition between 
school and post-compulsory education, transitions between different modules, 
institutions, or years of study, and transitions from study to employment. In 
each case, there are challenges for disabled students as the support mecha- 
nisms, strategies, and expectations placed upon them may change. The design 
of technology and technology-related support can be a pivotal factor within 
this (Burgstahler, 2003). Examples of challenges raised included the removal 
of assistive technology that was loaned or supplied by one organisation as the 
person transitions into the remit of another, or the incompatibility of work- 
place systems with the assistive technologies and strategies that the person has 
developed as a student. 

Attention should be drawn to resolving these types of gaps that emerge as a 
person moves through a transition. At the same time, it was argued that it is 
essential to support the development of self-advocacy - an individual’s skills 
and capacity to describe their requirements, and the confidence to know their 
right to reasonable adjustments to support them. The experiences of network 
members, and prior research, both highlight that developing a persons’ capac- 
ity for self-advocacy plays a pivotal role in successful transition, because there 
is often no single consistent entity supporting them across the transition. 

By bringing together students, practitioners and researchers, complex issues 
can be unpicked with discussion across stakeholders. One issue in which all 
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voices are required to build understanding is the adoption or rejection of tech- 
nologies by students. In many countries or institutions, resources are spent 
on making specific assistive technologies available for students and providing 
training for these. However, research by network members has found substan- 
tial disparities between the technologies that experts suggest are useful and 
those students consider useful (Fitchen et al., 2013). Other work by network 
members has explored related issues, such as how mainstream and specialist 
assistive technologies can both be useful in making learning accessible (Seale 
& Cooper, 2010). In the Ed-ICT approach, students presented their views of 
which technologies were useful to them and how they chose and used these. 
Practitioners, including those supporting disabled students in education and 
technology developers, expressed their perspectives on how they supported 
students, and researchers presented findings and provocations. The proceed- 
ings of these workshops then became a basis for balanced analysis and agenda- 
making for continued work that includes all these perspectives (Ed-ICT, 2017; 
Ed-ICT, 2018a, Ed-ICT, 2018b). Proposals for further research or practice- 
based innovations can emerge that combine the different potential, resources, 
and expertise of multiple local contexts. 

Each of the approaches outlined in this section - The OER World Map, 
GO-GN, OWL, and the Ed-ICT Network - provides a different example 
approach to how collaboration can respond to the potential for both global and 
locally situated research. They each build on an awareness that local context 
matters in terms of the available support for accessing learning opportunities, 
and in the ways in which research can have an impact. They also harness the 
value of openness through global networks. 


Harnessing data to understand barriers 
and improve support for learning 


While the increase of interactions with technology creates the potential for ever 
more data to be collected and analysed, this does not necessarily lead to greater 
understanding of the needs of learners. 

Prioritising openness and accessibility does present challenges to big data or 
analytical research approaches. For example, in the Bridge to Success initiative 
introduced in the previous chapter, it was noted that if we aim to create a situ- 
ation in which anyone can access and use a course or resource in flexible ways, 
we cannot then put restrictions on them such as requiring them to fill in details 
about themselves. Neither can we necessarily gain access to institutional data 
about students in order to use this to contextualise and understand their learn- 
ing (Pitt et al., 2013). In Bridge to Success, we worked closely with some partner 
colleges to evaluate the impact of introducing the OER into particular classes in 
their institutions, but in other scenarios, OER can be adopted and used with very 
little trace of this activity having occurred, or a sense of its impact on learners. 
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Equally, supporting the flexibility required for disabled learners can also 
challenge analytics approaches. For example, if a learner requires their learning 
materials in an alternative format, they may not produce data through their 
interactions with a Virtual Learning Environment (VLE) in the same way that 
other learners would. Ifa research method uses data on VLE interactions, such 
as measurements of student logins or page views, are students using alternative 
means of access and engagement represented in the data? These issues need 
attention for the benefit of both the students who engage through these means 
to ensure they are not excluded, but also for the validity of the research, which 
might otherwise inaccurately suggest a lack of engagement with the learning 
materials where actually these are being delivered by alternative means. 

While these problems need to be considered, data-driven approaches, which 
are explored in Chapter 7, can also be used to understand the accessibility of 
courses, and to identify where potential problems might exist. Cooper et al. 
(2016) conducted an analysis using a large data set spanning five years of mod- 
ule-level data on student retention. By analysing the proportions of students 
with declared disabilities who completed each particular course or module, they 
could identify discrepancies where disabled students were performing more 
poorly than could be expected, using odds ratios of the likelihood of completing 
a module when compared with students who had not declared a disability. 

Cooper et al. (2016) note that this approach only identifies modules where 
there may be accessibility problems. It does not tell us what the causes of these 
lower completion rates in these modules are. Therefore, they also explore the 
use of student feedback to augment the approach. Feedback is commonly col- 
lected from students on all modules through end of module surveys. If this 
contains free text responses, and if the responses within this data that relate to 
access issues can be isolated, this feedback offers a means to create improve- 
ments which follows the notion that accessibility should be considered as a pro- 
cess (Coughlan et al., 2017). For Cooper et al. (2016), their analysis of student 
survey feedback highlighted a different set of courses that may contain acces- 
sibility issues to the ones they identified through the comparison of completion 
rates. This suggests that multiple approaches to evaluating course accessibility 
are complementary rather than providing similar results. 

A focus on one particular variable, such as whether or not a student has 
declared a disability, offers potential for insights such as those found for course 
accessibility by Cooper et al. (2016). However, students and the challenges they 
face are not one-dimensional. The concept of intersectionality - that multiple 
aspects of discrimination co-exist and interact - should also be considered in 
analytical strategies. By analysing multiple intersecting data points about a stu- 
dent (for example, gender, disability, socio-economic status and race), we can 
harness data to identify and explore the combined effects (Borden & Coates, 
2017). Encouraging research and evaluation with an intersectional approach 
is now an explicit focus for the Office for Students, the body responsible for 
Higher education provision in the UK (Office for Students, 2019). 
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Noting that most prior research had focused only on one or two demo- 
graphic variables, Rizvi et al. (2019) set out to analyse the relative effects of six 
demographic variables on the attainment (distinction, pass or fail) of students 
who had completed one of four OU courses. These variables comprised geo- 
graphical region; socio-economic status via the Index of Multiple Deprivation 
(IMD) for the student’s postcode; highest level of previous education; age, gen- 
der and disability. Using a Decision Tree analysis approach, they identify that 
the geographic and socio-economic factors had the largest impact of all these 
factors for these students. Analysis such as this is important to ensure a broad 
understanding of factors impacting on attainment. 

End-of-module surveys and data collected through student behaviour and 
assessment are the current materials available for learning analytics, but there 
could be better methods for students to self-report their experiences in ways 
that provide further insights. Prompted by the desire to support disabled stu- 
dents to represent the challenges they faced and the impacts of these on their 
studies, the Our Journey tool (https://ourjourney.ac.uk/) has been developed 
with the participation of students and other stakeholders to provide a struc- 
tured approach to capture the diverse journeys our students take (Coughlan 
et al., 2019a). By creating a series of ‘cards’ that represent important events for 
the student and their emotions at the time, combined with free text to further 
contextualise the event, we can create a different means of representing the 
student experience. 

The representation of a journey taken over time helps to unpack the ways in 
which series of events and the development of the person combine in both the 
challenges and achievements of study, and the impacts other areas of life may 
have on study. Our Journey aims to capture the experience of each student, 
but we are exploring how this can be combined with other forms of analyt- 
ics data and events. In this way, adding additional context and narrative that 


Tim Coughlan, tell us about your journey 


What happened? Assessment ’ How did you feel? anxious g 

Select a word or an Select a word or emoji 

event connected to 2) Em which describes how 

your journey < 4 > you felt. < o o > 
v 
x 

Describe the event (optional) When did it happen? (optional) 

My first exam! Autumn 2019 


Add any word or date format you like 
For example: 5 November 2019 | November 
2019 | Autumn 2019 | 2019 | 2018 


186 characters remaining 


POST 


Figure 5.2: Our Journey card creation interface. 
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Figure 5.3: A student journey representation created in the Our Journey tool. 


is structured by the student rather than the institution. Furthermore, unlike 
a survey, Our Journey is designed to be engaging and enjoyable for the stu- 
dent, with the potential to underpin reflective learning activities around study 
skills. Finally, options are being explored to integrate the tool, with prompts 
for guidance and support. We continue to iteratively refine the design, and 
to trial the tool in a variety of ways, in order to develop an evidence base 
and improvements. 

Our Journey is being developed with continual input from students and 
staff, and was informed by our prior research around the challenges that disa- 
bled students face and the impacts of these (Coughlan & Lister, 2018). One 
particular area that was highlighted in this process was the importance of the 
emotional effects of events in the student journey. For example, that having to 
complete arduous administrative processes is a cause of stress and potential 
exacerbation of mental health challenges. As such, the application of Our Jour- 
ney to understand student mental health and wellbeing is an important direc- 
tion in our work. Because students report an emotion in relation to each event, 
and the patterns in this over time can be studied, the tool has the potential to 
uncover patterns and types of events around emotional wellbeing. 

More broadly, there has been a wave of activity around student wellbeing 
and mental health in recent years. This has been prompted by data showing 
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increased disclosure of mental health conditions, and high-profile cases of 
student suicide. As well as providing new understanding at scale of student 
mental health, analytics and other technologies could underpin preventative 
approaches that respond to students with relevant guidance and prompts to sup- 
port (Jisc, 2019). However, the complexity of practically measuring or engaging 
with student mental health requires qualitative, co-produced approaches to the 
design of strategies (Piper & Emmanuel, 2018; Piper & Byrom, 2017). 

Future advances in our capacity to understand our students and the chal- 
lenges they face as they access education may depend on our ability to combine 
analytics drawn from university systems with the participation of students and 
other stakeholders to give context. 


Innovation for inclusion benefits everyone 


Rather than inhibiting innovation, a focus on inclusion is key to directing 
innovation towards human-centred outcomes that are useful for all people. By 
embedding accessibility from the start of the process, and by working in the 
open, we create greater opportunities for people to use and build on our work. 

The ‘Our Journey’ tool described above provides a reminder of this. The pro- 
ject developed because we recognised a need for disabled students to be able 
to communicate the challenges they face, as well as a difficulty for educators, 
researchers and support staff to understand these challenges. 

However, we now see that Our Journey could be useful for all students, and 
that the underlying concept has potential for a wide range of scenarios. This is 
because most students face some challenges and could benefit from representing 
and reflecting on these, as well as their successes. Equally, institutions lack rich 
understanding of their students in general, so could benefit by understanding the 
journeys of all their students. There is wider interest in applying the Our Journey 
concept even further from this starting point, by using the activity of creating a 
journey as a way to capture and learn from personal experiences in a range of 
different domains. The tool and graphics are openly licenced, which can simplify 
and enable adaptations of the tool to different types of activities and contexts. 

This doesn’t mean that the project has become detached from the original 
purpose - we maintain the involvement of disabled students and will still use 
the tool to create greater understanding of their experiences. But rather than 
suggesting that accessibility and inclusion constrain innovation, projects like 
Our Journey identify needs or goals by working with a particular population, 
and direct innovation towards it (Coughlan et al. 2019a). In doing so, it is 
often the case that the goal which is particularly apparent to this population is 
actually more widely applicable. By aiming for a mainstream audience, a tool 
that is inclusive by design is no longer a specialised solution. Instead we have 
empowered the underserved audience such that they are directing innovation 
in the mainstream. 
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It is argued in the Beyond Prototypes framework that “TEL involves complex 
systems of technologies and practices” (Scanlon et al., 2013, pg. 7). Because 
inclusive innovation projects are grounded in tackling real problems, they are 
usually open minded towards combinations of technological and practice- 
based solutions. For example, the “Returning to STEM’ Badged Open Course 
(BOC) was based on research that explored the challenges for women return- 
ing to STEM careers after an extended break (e.g. because of child care). The 
project identified strategies that had been successful for women returning to 
their careers. BOCs are a model for free courses designed to support independ- 
ent study (Law, 2015). The ‘Returning to STEM’ BOC was created by drawing 
on the project research and through a partnership with Equate Scotland - an 
organisation that works towards the advancement of women in STEM careers. 
However, a key lesson learned was around the effectiveness of a blended learn- 
ing approach, which combined the BOC with face-to-face workshops and indi- 
vidual meetings (Herman et al., 2019). While there is a tendency to focus on the 
online experience of OER, it can be at their most useful when combined with 
face-to-face teaching and support. As Cannell and Macintyre (2017) argue, 
partnerships that provide for physical, face-to-face activities to introduce 
learners to OER and build confidence are important because there is a danger 
that if we only make online learning opportunities, we are likely to reinforce 
digital and educational divides. 

Where technological advances are developed to improve inclusion, these 
often go on to underpin mainstream advances. Captions and transcripts for 
online videos provide a useful example. Back when online learning was still rel- 
atively new, Colwell et al. (2005) described the development and evaluation of a 
video player to identify requirements for deaf students. This supported existing 
transcripts to be displayed alongside the relevant video, and for software-gen- 
erated transcripts to be produced where there was no transcript available. We 
now find that many students benefit from such transcripts in situations where 
they cannot easily listen to audio or prefer to read (e.g. reported in Coughlan 
et al. 2013). More broadly, the technologies that developed for speech to text 
(speech recognition), and text to speech (screen readers) as an essential com- 
ponent of assistive technologies now find both in pervasive use in mainstream 
technologies from smart speakers such as Amazon Echo, smartphone-based 
assistants such as Apple’s Siri, and in automated telephone answering services. 

The value of innovation also flows in the opposite direction, with mainstream 
technologies having the potential for specialist assistive uses. If properly har- 
nessed, virtual assistants, smart home devices, and other Internet of Things tech- 
nologies can be beneficial for inclusion. Technology companies, such as Microsoft, 
now recognise that Artificial Intelligence (AI), combined with pervasive mobile 
computing, has many potential applications to accessibility (Microsoft, 2019). An 
example of this is the ‘Seeing AT app, which provides visual recognition of objects 
and reading of text in the environment using a smartphone camera (Microsoft, 
2018). By focusing on the development of an innovation that supports blind 
or low vision people, they provide a grounded and important challenge for the 
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underlying technology to be applied to. There is, however, a perennial concern 
that if these new technologies are not designed with consideration of accessibility, 
they may instead exclude by design. The development of standards through global 
networks is again important here (Abou-Zahra et al., 2017). 

While some AI innovations will be specialist assistive technologies, the 
integration of accessibility with mainstream technologies means that people 
are not segregated or left out of activities. Live automated captioning of lec- 
tures or other presentations is being integrated into mainstream presentation 
software such as Microsoft Powerpoint. It seems likely that more teachers 
will use this technology than if a specialist tool needed to be purchases and 
installed. The Android Live Transcribe app offers immediate speech to text 
conversion as a means for deaf or hard of hearing learners to communicate as 
equals with peers in collaborative activities, just using a standard smartphone 
(Android, 2019). 

Taking this idea further, AI is being applied to overcome communication bar- 
riers of all kinds for all people, automatically translating audio and text between 
any language that is spoken, including sign languages (Ahmed et al., 2018; 
Wolfe et al., 2016). We are at the point where it is possible to translate, for exam- 
ple, British Sign Language into written Spanish and vice versa. To achieve this, 
sensors to convert gesture into data, text to speech, translation services, speech 
to text, and virtual reality technologies to enact a signing avatar would be used 
in concert. As these various technologies mature, the potential to create online 
learning that is more global and inclusive of all groups becomes a realistic and 
exciting prospect. 

Some people may expect innovation to work best when unconstrained by 
the hassle of having to produce results that work for everyone. But meaning- 
ful innovation should identify and work to tackle real issues. In our current 
research, we are working with Microsoft to explore how an AI-based assistant 
could support people through the processes of communicating about their 
disabilities and getting effective support in study and everyday life. Research 
grounded in the participation of students identified these issues (Coughlan & 
Lister, 2018), and now provides an inspiration for us as we explore how to har- 
ness and innovate with these technologies. 

Having argued that the results of inclusive innovation projects are likely to be 
useful to wider audiences, we want these to be adaptable and easily available to 
others to use. This leads us back to importance of an open approach. The tool 
and graphics are openly licenced, and our discussions about adapting the tool 
to different types of activities and contexts are made easier because of this. 


Taking advantage of opportunities: 
bricolage and guerrilla research 


In the previous chapter, we described persistent strands of research in the 
areas of virtual laboratories and remote access to fieldwork. These pushed the 
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possibilities of technology at the time, and there was value in conducting these 
experiments to produce a close approximation of what would become more 
achievable in time. Taking available opportunities to use cutting edge technolo- 
gies in new ways allowed for the investigation of pedagogical possibilities, in 
advance of these becoming a mainstream reality. We use what is available now 
to create and learn things that should be important later. 

Taking these opportunities as they arise is important to the bricolage 
approach. As Scanlon et al. (2013) put it, bricolage ‘involves bringing together 
and adapting technologies and pedagogies, experimentation to generate further 
insights, and a willingness to engage with local communities and practices. (pg. 
7). Key features of the approach are that a project may start by reviewing what 
tools and resources are available and how they could be innovatively used; and 
that the use of theory to underpin research is balanced with engaging commu- 
nities and grounding innovation in practice. Attention is paid to the constraints 
of a situation and how these can be overcome or compensated for. 

Bricolage can be particularly pertinent to work around inclusion because of 
the pressing need to address and have an impact on real problems faced by peo- 
ple. Also, exclusion often occurs through restrictions and constraints within 
particular situations, so attending to these is often an effective starting point 
and continued interest for any project. 

We also see the value of bricolage in the more recent IncSTEM project 
(Embedding and Sustaining Inclusive Practices in STEM). This built on 
and scaled up existing examples of inclusive practice in STEM at a range 
of levels, including teaching activities, institution-wide systems and policy 
and sector-wide accrediting bodies (Pearson et al, 2019). In order to do this, 
IncSTEM has sought the voices and involvement of staff and student stake- 
holders from across the university and the sector, adopting a diverse range 
of methodologies and a collaborative approach (McPherson et al, 2019). The 
aim here is to systematically take opportunities to review, refine, and spread 
innovations for which there is evidence that they could make a difference 
to inclusion. 

The potential of open approaches to support and help to spread innovations 
widely in education is commonly alluded to, but as noted in the Beyond Pro- 
totypes framework, such processes need to be viewed longitudinally, which is 
problematic when projects are only funded for short periods. In a retrospective 
analysis of Bridge to Success, which returned to interview stakeholders in the 
three years following the completion of the project (Coughlan et al. 2019b). 
Through this we found instances where the introduction of the OER to new 
audiences led to wider change, such as embracing the idea of providing free and 
openly licenced materials for all students studying with the institution, rather 
than expensive proprietary texts. Equally, enthusiasm for OER by individuals 
could be tempered over time by a lack of organisational buy-in, and the with- 
drawal of the support that the project funding could enable. We must conclude 
that sustainability is an essential focus that can be hampered by the project and 
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innovation focused world of TEL research. Once again a persistent intent in an 
area means that successive projects build on each other. 

The concept of bricolage can also help us to understand how an open resource 
can become a basis for innovation. In the case of Bridge to Success, the origi- 
nal course materials, such as ‘Starting with Maths, were designed to support 
learners new to higher education with support from a tutor. Years after their 
original production, these course materials were openly-licenced and remixed 
for self-directed use, predominantly for a US audience. This created ‘Succeed 
with Math. Additional elements such as quizzes were added which supported 
independent use of the materials, and further revisions of these resulted in 
‘Succeed with Maths’ Parts 1 and 2. These were some of the first set of Badged 
Open Courses (BOCs) referred to earlier. This potential for reuse and remixing 
of tried and tested educational content in new contexts helped to make innova- 
tion a reality (Coughlan et al. 2019b). 

If we consider what openness offers as part of an approach to bricolage, there 
is also a sense of supporting opportunities to arise and taking advantage of the 
increasing resources that are freely available to use. Valuable research projects 
can be constructed using open data, open source tools, platforms and people 
who can engage, or openly-licenced materials. 

Arguing for the value of harnessing this, Weller (2014) describes the notion 
of ‘guerrilla research, as an alternative to the common template of an academic 
research project. The key notion is that in many situations where we identify 
a research question, there are open resources that provide opportunities to do 
interesting research right away, for ourselves, and without extensive costs and 
planning. Guerrilla research can have the following characteristics: 


e It can be done by one or two researchers and does not require a team. 
e It relies on existing open data, information and tools. 

e It is fairly quick to realise. 

e It is often disseminated via blogs and social media. 

e It doesn’t require permission. 


These characteristics can be seen in initiatives to harness open data as a means 
to social change. A nice example of this has been the School of Data initia- 
tive, which has created structures to enable small and large projects around 
the world that develop data literacy among journalists and NGOs, and lead to 
practical results by exploring and creating publicly available data (School of 
Data, 2019). 

While longer term plans and funding are important for many research 
projects, these may become barriers that prevent progress being made in 
the development of knowledge. One of the great things highlighted by this 
approach is the way in which it promotes the notion that anyone can conduct 
research at any time, we just need an idea or question that matters to get 
us started. 
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Conclusions 


Continual change in technology and in education mean that inclusion requires 
constant re-evaluation and discussion. In this chapter we have explored some 
of the practices and trends that are important to the present and future of our 
work in accessibility, openness and inclusion. 

We have highlighted how global networks can support sharing of research 
and practice and work towards shared goals. In the modern age these networks 
are always likely to have a digital foundation, and these networks can embody 
and exemplify how open practices enable wider participation and inclusivity. 
However, these networks should still be designed to account for individual 
and local matters. Networks based around principles of openness and inclu- 
sion have created global standards, but they also help localised activities gain 
traction, and provide individual researchers with homes and communities that 
benefit their research and enhance the impact of it. 

From the early foundations, data analysis has been used to identify gaps in 
participation, completion and attainment for particular groups. More recently, 
the trajectory of innovation in data gathering and analysis has been to embrace 
complexity by looking at wider ranges of variables and intersections between 
these, and to develop means to capture and analyse events in the student jour- 
ney and their impacts over time. These analyses often lead to more questions, 
and there is still much that we do not know about the mechanisms of exclusion. 
But we are embracing new opportunities, including more data captured from 
learners interacting with online learning environments, data at scale from new 
platforms for open education, and new learning analytics tools and techniques. 

We are also finding that our work to address audiences with particular needs 
leads to wider impact. There is increasing recognition that harnessing tech- 
nologies such as AI to address problems identified by working with a minor- 
ity group is an important means to create mainstream innovations. Rather 
than considering these outcomes as incidental, we can argue that inclusive 
research and innovation should be the norm. This is achieved by opening up 
our projects to participation at all stages, and making sure that the outputs of 
these are available to others to innovate with through their own bricolage and 
guerrilla research. 
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CHAPTER 6 


Evidence-Based Learning: Foundations 


Ann Jones, Bart Rienties and Canan Blake 


This chapter discusses some of the Computers and Learning (CAL) research 
groups early work, focusing on our attempts to understand learners’ practices so 
that teaching could be adapted to meet learners’ needs. The chapter describes and 
discusses examples of CALRG research from the group’ early days to the start 
of the 2000s. One reason for doing this is to explore the extent to which there 
has been continuity in the groups work over time. In the chapter we argue that 
the groups motivation, aims, ethos and overall approach have remained simi- 
lar during its forty-year existence. The chapter draws on the Beyond Prototypes 
framework, described in Chapter 1 of this book, to frame some of the discussion, 
in particular focusing on policy and environment. Analysis of the case studies 
that led to the development of the framework suggest that Technology Enhanced 
Learning (TEL) needs to be understood as a complex’, made up of a series of ele- 
ments that need to be considered together. The chapter also uses the three themes 
of the group’ first conference to provide an organising framework for the discus- 
sion. The three themes from that first conference are firstly, models of learning; 
secondly, methods for studying learning and thirdly, institutional research. 


Introduction 


As will be highlighted below, much early CALRG research was experimental, 
and ground-breaking at the time. While many students are now used to working 
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with computers, smart-phones, and tablets, and may have only lived in an era 
where CAL was always around, it may be useful to remind the reader that 40 
years ago most students did not have access to computers, let alone the Internet. 
For example, only 13% of households in the UK in 1985 had a home computer, 
and only in 2002 did a majority of households have at least one home computer 
(Office for National Statistics (UK), 2019) Similarly in an age of ubiquitous con- 
nection to the Internet, it seems hard to remember a time when people were 
not connected. In fact, in 1998 only 9% of UK households had access to the 
Internet, mostly using a slow telephone modem for those who remember. Only 
in 2005 did more than half of UK households actually have online access, which 
is nearly 25 years after CALRG was established. Therefore, some of the case- 
studies discussed will need to be interpreted in their historical context. 

This chapter will discuss some of the research group’s work, focusing on our 
attempts to adapt teaching to meet learners’ needs. This will serve to illustrate 
and document some of the research that has taken place from 1979 to the 
2000s. It will also support our argument for continuity in the group’s work over 
time: that the motivation, aims, ethos and overall approach of the group during 
its forty-year existence have remained similar across the years. In discussing 
the work, we will draw on three aspects of the Beyond Prototypes framework 
(policy, environment and funding), and will also refer to the three themes of 
the group’s first conference to provide one organising framework for the discus- 
sion. These three themes are firstly, models of learning; secondly, methods for 
studying learning; and thirdly, institutional research. The Beyond Prototypes 
framework developed by Scanlon et al. (2013), has been described in the first 
chapter of this book. The case studies that led to the development of the frame- 
work suggest that Technology Enhanced Learning needs to be understood as a 
‘complex’ made up of a series of elements that need to be considered together, 
as represented in Figure 1 (Scanlon et. al., op. cit.) and reproduced in Chapter 1. 
In this chapter we will illustrate how the elements of the complex have been 
applied by the CALRG in our research into our students’ learning and trying 
to meet their needs. 


Models of Learning: a cognitive science approach 
to understanding the learner 


As noted in the introductory chapter, during its first decade much of the CAL- 
RG’s work was in the area of cognitive science. There was a strong interest in 
applying this to the OU context through considering how instruction could 
be designed to help improve student performance, so the relationship with 
teaching was strong. Alongside this was an interest in theory development, 
e.g. understanding how problem solving skills were developed. And again as 
described in the introductory chapter, one focus was on developing under- 
standing of learners’ practices through collecting student protocol data. 
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One example of this approach is Jones’s doctoral research on novices learn- 
ing programming (Jones, 1993). Four different programming languages in use 
at the OU were investigated, including high level and low-level programming 
languages. The first example is of learning SOLO: an AI programming language 
designed by Eistenstadt (1982, 1983), to introduce Open University cognitive 
psychology students to Artificial Intelligence as a tool for modelling human 
cognition. The aim was to make it easy for the user to get the system to do what 
they wanted it to without getting tangled up in trivial spelling and syntactic 
errors. SOLO was designed so that such unproductive errors could be trapped 
thus optimising productive interaction. The main component of SOLO is a lan- 
guage for manipulating a relational data-base, containing facilities for inserting 
descriptions into the database and for pattern matching against descriptions 
already in the database. 

Eisenstadt (1982) explains the motivation behind SOLO in greater detail and 
also the project that included a “six year period of design, implementation, test- 
ing, and iterative re-design of a programming language, user-aids, and cur- 
riculum materials for use by Psychology students learning how to write simple 
computer programs’ (Eistenstadt, 1982, p.1). 

Thirteen participants took part in the SOLO case study, all of whom were 
studying the cognitive psychology course and had agreed to come into the OU 
psychology laboratory when they reached the part of the course where they 
were starting to learn SOLO. 

The participants in the OU laboratory worked through the instructional 
materials in the SOLO book. They sat in a room on their own to work and 
were recorded talking aloud about what they were doing - the researcher 
would go in from time to time and ask how they were getting on. The task 
that students were engaged in was working through the instruction book and 
they reached the first activity that required them to produce some code, the 
so-called ASSESS problem. This problem was described in the course booklet 
as follows: 


“Define your own procedure called ASSESS which prints out UNHEALTHY 
if someone (the node to which it is applied) either drinks whisky, on the one 
hand; or else if that person both smokes cigarettes and drinks beer. Using 
the NOTE procedure, add some descriptions of your own to SOLO’ data- 
base, and try out your ASSESS procedure to get it working properly. You 
must decide for yourself how you are going to represent “drinks whisky” 
etc. in the database? (Eisenstadt, 1983, p56.) 


Some further context will be helpful to make sense of the protocol data. The 
SOLO primer which participants worked through provides two particular 
examples to illustrate how flow of control works in SOLO which is described 
in a section entitled “Sequencing of programs. (Eisenstadt, 1983, p.54). These 
examples are referred to as the ‘weakassess’ and ‘strongassess’ procedures. 
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So what | did was to combine the weakassess and strictassess type programs here, so, here we've 
got line 10 if drinks whisky, print unhealthy and exit. If absent continue. ... 

That's the weakassess model. 

The next two takes part of the strictassess model, so...: somebody has to smoke cigarettes and drink 
beer to be deemed unhealthy, so therefore line 20, check somebody smokes cigarettes if absent print 
healthy 

I didn't know if it was right to do healthy and unhealthy, - it doesn't specify that. 

So, if present continue and it goes to line 30, check drinks beer, if present print unhealthy because by 
definition if it's got to drinks beer it's had to have been smoking cigarettes as well. If absent continue 
and then at line 40 print healthy. 


Figure 6.1: an example of protocol data from one of the students, Jane. 


These examples are referred to by Jane, one of the participants, as she works 
on defining the ASSESS problem. Figure 6.1 shows part of Jane’s explanation 
of working on the ASSESS procedure. She drew on examples in the book 
- e.g. referring to the ‘weakassess model’ here, and indeed she worked it 
out correctly. 

Other students had more difficulties. These were particularly apparent when 
they were studied learning a different kind of programming language - an 
assembler language. This is a level of programming languages where there is a 
strong correspondence between the program’s statements and the architecture's 
machine code instructions. 

This study by Jones (1993) indicated the importance of data which pro- 
vided information about how students interacted with and acted on the text, 
and the extent to which the design of instructional materials supported their 
learning. For some of the analysis programs were approached and viewed as 
a collection of plans, and this helped to identify the extent to which learn- 
ers identified (or did not) and used (or did not) appropriate plans. For a 
researcher and teacher to observe directly how students react to materials 
that have been written, or technologies that have been developed is a very 
powerful experience. 

A different, later approach to observing learners’ behaviour and interactions 
with computers was the establishment of the data capture suite, many years 
later when more sophisticated technology could be deployed but with a similar 
aim. This was to observe and capture detailed learner interactions with media 
- although by this time the CALRG group was focussing on students’ inter- 
actions with computers rather than text. One report is by Blake and Scanlon 
(2003) who used video data to analyse collaborative learning in what became 
known as the ‘data capture suite. 

Learning design has developed considerably since the early work. The next 
chapter charts the development of the OU learning design initiative and dis- 
cusses current research into the relationship between learning design, student 
behaviour, satisfaction, and performance. However, like the early work, there 
is still an emphasis on detailed information about student interactions with 
course materials. 
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In undertaking the observation work carried out in the data capture suite, 
Blake and Scanlon’s overall enterprise was to the investigate “the usefulness 
of technology-mediated collaborative problem-solving as part of an ongoing 
research programme.” (Blake and Scanlon, 2003, op. cit., p5.) For this series of 
studies, the emphasis had shifted away a little from The Open University’s stu- 
dents, although the authors note how the work is associated “with a desire to 
improve the experience of learning for our students”, Blake and Scanlon, 2003, 
p.5. They also refer to their use of the CIAO! Framework, developed within the 
CALRG for evaluating CAL, and how it draws on a variety of sources, using 
both qualitative and quantitative approaches (Jones et al., 1999). In evaluating 
student use of computers, particularly collaborative learning, they argued for 
the need to observe students interacting with the educational innovation, and 
also note Issroff’s holistic approach (Issroff, 1995) which in addition to record- 
ing interactions emphasises the importance of affective measures. Issroff et al., 
(1994) analysed students’ collaborations over a number of sessions and the 
results showed developments over time. In their overview of this work Blake 
and Scanlon (op. cit., 2003, p.6) advocated the use of video data: “Examining 
the interactions that students have with computers and with each other requires 
observational data, preferably supported by video data”. 

They note that the advantages of such an approach include: 


e Its relative objectivity. 

e That analysis can be carried out collaboratively by more than one researcher. 

e Its use for either or both qualitative and quantitative data. 

e That considerable amounts of data can be stored and analysed relatively 
easily by video-analysis software. 


The data capture suite was developed to enable video capture of interactions 
and combined video data records of each participant with a synchronous 
record of their computer screen. 

This approach was used to investigate a range of problem solving and learn- 
ing tasks including: teenagers learning the laws of momentum (Whitelock and 
Scanlon, 1996), children learning about the phases of the moon (Whitelock et 
al., 1996); adults learning applied maths (Smith et al., 1989), and healthcare 
professionals using CoMET (Concept Modelling Environment for Teachers) 
to investigate the educational potential of a concept-based toolkit (Alpay and 
Giffen, 1998). 

One study (Scanlon et al., (2000) investigated the problem-solving behaviour 
of pairs of adults working on a statistical problem. As in the earlier studies, pro- 
tocol data was gathered, but additional video data made it possible to observe the 
subjects’ non-verbal gestures. The video provided evidence about the degree of 
certainty with which the participants put forward their suggestions or solutions 
to each other and also recorded their reactions to their partner’s suggestions. In 
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comparison, a verbal protocol does not always contain clues about these behav- 
iours. This example is part of a larger database with which the group explored 
the value of videoconferencing and eye contact during remote problem-solving. 
The study established that pairs who communicated with video which enabled 
eye-contact were more successful in their problem solving. 

Later work, often led by CALRG research students, included an investigation 
of how newer technologies (newer at that time) might be employed for identi- 
fying learners’ attention, recording real-time writing and sketching, and ana- 
lysing multiple data feeds in an integrated way (San Diego et al., 2012). This was 
a study of learners’ interactions with multiple representations to illustrate the 
advantages and disadvantages of digital approaches to collecting, coordinating 
and analysing observational data. In these investigations detailed gaze videos 
were obtained and were able to indicate the paths as well as ‘fixations. This 
allows researchers to study participants’ attention in detail and how it changes 
over time (see San Diego and Aczel, 2007). 

Other research used protocol analysis again combined with a more quantita- 
tive approach as a way of observing students using software via the internet 
(Hosein et al., 2007), thus providing remote observation. In this approach, stu- 
dents used a remote application facility on their own computer to connect to 
the researcher’s computer: they were then able to interact with this computer 
and use software on it. Audio and video data, mouse clicks and keyboard entry 
were captured. A quasi-experimental design was used for collecting mainly 
quantitative data but by adding on talk-aloud strategies, interviews and video- 
ing, qualitative data was also collected. As the researchers noted, this approach 
to understanding students’ use of software for problem solving is not limited 
to studying students in a particular setting but to any student connected to the 
Internet in an environment where rich qualitative and quantitative data can 
be collected. 

These studies, where there is an emphasis on the detailed analysis of interac- 
tion among learners, can be seen as having a learning analytics focus, although 
that field had yet to emerge. 


Evaluating CAL programs: institutional research 


Collecting protocol data was one method for studying student learning, which 
as noted above, was used over a long period of time. Other approaches were 
taken in the evaluation work that the CALRG conducted, and this is discussed 
in this section. 

Early evaluation work in the CALRG was on understanding student use 
of particular CAL programs including CICERO, (Jones and O'Shea, 1982); 
Works Metallurgist, (Blake et al., 1996); MERLIN, CALCHEM and EVOLVE 
(Scanlon et. al., 1982). These CAL offerings were developed in response to 
student needs or challenges. For example, the CICERO CAL tutorials were a 
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way of providing diagnostic feedback and additional help to students. The aim 
was to understand student behaviour and improve teaching. It is important to 
note that as courses were often in place for eight years or more, and the main 
component was printed text, it was not possible to make changes to the text 
following student feedback. However, it was possible to make changes to the 
CAL programs. 

The evaluations aimed to understand the extent to which students used 
such CAL; benefits and challenges; how they used them and how the pro- 
grams might be improved. Two case studies will illustrate the CAL evaluation 
studies, the first one conducted was the CICERO evaluation (Jones & O’Shea, 
1982). After each case study summarising the research we will offer some 
brief reflections. 


Case study 1: Tutorial CAL in the early 1980s 

The first study focused on tutorial CAL: (Jones & O'Shea, 1981; 1982). The 
main aim of these programs was to provide diagnostic feedback, remedial help, 
and revision aid. The particular tutorial CAL program evaluated in this study 
was called CICERO, and first used on a psychology course in the Educational 
Studies faculty: “Personality and Learning’, in 1977. Note that this was before 
the establishment of the CAL research group - and was one of the first, if not 
the first, evaluation studies carried out by the CALRG - motivated by a desire 
to understand more about student use, or lack of use, of CICERO. 

CICERO was available at study centres across the four nations (where tutori- 
als were held), and there was a less interactive postal version too. Study centres 
were not open all the time, so students needed to check that they would be 
open, and once there they would be using the tutorial via a terminal. 

For each tutorial, diagnostic questions relating to a specific block of the 
course were sent to students to answer at home; the answers provided informa- 
tion about students’ conceptual strengths and weaknesses related to the specific 
objectives of the block and course. These answers were taken to the study cen- 
tre, the program accessed, and the answers typed in. Further questions might 
then be asked and according to the answers, advice and remedial help would 
be given. A ‘postal version was also available providing advice based on the 
students’ performance on the diagnostic questions. The student would receive 
a printout a few days after posting the answer form. 

Use of the system on three courses was rather low and dropped during the 
course of the academic year, so it was decided to evaluate the use on one course 
- the interdisciplinary course Biological Bases of Behaviour where 4 CICERO 
tutorials replaced 4 computer marked assignments. The study aimed to find out 
why students used or failed to use the tutorials and their beliefs about the edu- 
cational benefits and practicalities. As the tutorials were optional, introduced 
no new material, and covered only a selected part of the course, there was no 
attempt to establish their educational effectiveness. The methods used consisted 
of an initial questionnaire; a questionnaire built into interactive tutorials and 
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sent with postal tutorials; interviews with students and staff at summer school; 
final questionnaire to follow up answers to earlier open-ended questions and 
tutor questionnaires. Usage figures from the computing service records were 
also available. 

Once students could access the tutorials the majority were satisfied and 
found they met their expectations, but the number of users fell rapidly 
throughout the year. What put students off? We found (as in many OU stud- 
ies): an instrumental approach to this optional study; a fear of secret assess- 
ment; fear of using computers and embarrassment at the possibility of making 
mistakes in front of other students. Hence we asked about these issues in the 
final questionnaires and 22% (out of 100) reported ‘bad computer experiences. 
The most prevalent bad experience was difficulty in access; of these the most 
frequently reported were logging in difficulties; 12% (of 543) reported they 
were nervous of using the terminals and 13% (again out of 543) talked about 
embarrassment. Only a small percentage (16%) intended to definitely use CIC- 
ERO again - the main obstacle was travelling and using it at the study centre. 
We ended up with a Chinese box of barriers, where each access issue is framed 
within the next: access to terminal (layer 1); access to program (2); quality of 
program and integration with the course. So students needed to negotiate a 
number of barriers, or layers, before engaging with the course tutorial itself. 
We noted that the real breakthrough would be in providing home access, and 
indeed the personal computing policy, described next, was set up many years 
later to provide such access. 


Reflections on case study 1 


Looking back at this study across nearly forty years, five elements struck us. 
Firstly, at a time when nearly all the focus of educational technology was on 
cognitive factors, affective issues were noted - students were concerned about 
secret assessment; fear of using computers and embarrassment at the possibil- 
ity of making mistakes in front of other students. Secondly, in terms of ‘analyt- 
ics, although no sophisticated records of use were available we did have usage 
figures available from the computing service records. Thirdly, the barriers were 
such that many students did not use the tutorial CAL - or they did not per- 
sist in using it - hence there was little feedback that fed back into the design 
of the programme. Fourthly, the approach taken in tutorial CAL (diagnostic 
multiple-choice questions) was a forerunner of computer-based assessment 
that developed significantly later especially in the science faculty. Finally, in an 
elementary way we were able to include some built-in evaluation (e.g.the ques- 
tionnaire at the end of the tutorial). The next section describes the university’s 
response to the barriers to access that were found in this and in many other 
CAL evaluations: the personal computing policy. 
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The home computing policy evaluation project 


As noted in case study 1, there were difficulties with accessing computers from 
study centres. Even so, use of the terminal access system expanded throughout 
the 1970s. It should be noted that most of the use was by students for whom 
access was a requirement. For example, from 1970, students on the math- 
ematical foundation course were required to spend around five hours online 
to the mainframe computer. During that period, similar students in traditional 
universities were also using computers through online, time-shared terminal 
access but the equipment was usually in computer laboratories on campus. By 
1980, students on 35 courses were using the OU system for some aspect of 
their study. This included computing courses, and courses began experiment- 
ing with various ‘standalone’ microcomputers at summer schools. Hence, Jones 
et al., (1993), referring to a period around the late 1970s, and the difficulties of 
access noted that: “despite these problems, student computing at a distance was a 
success and there was pressure to expand” (p.42). 

This led to the development of the “Open University’s Home Computing 
Policy” that both required students (on certain courses) to acquire their own 
computers and supported them in doing so. This was a large-scale innovation, 
affecting 17,000 students by 1992. Running alongside the policy development 
was a research project that evaluated the policy. The book describing this edu- 
cational evaluation explains that “We set out to investigate the effects of requiring 
students on particular courses...included in the policy, to make their own arrange- 
ments for acquiring a microcomputer”. (Jones et al., 1993, Preface). In terms 
of the TEL complex, taking account of the ecology of practices and technical 
content is particularly salient in the Home Computing Policy (HCP) project. 

The ‘success’ of student computing at a distance meant an increase in the 
number of courses that wanted to include some form of computer provision. 
Although terminals and the mainframe were updated, the university system 
could not even cope with student demand from existing courses. Courses began 
to experiment further with using ‘standalone’ micros at residential schools: the 
evaluation of one such experiment is reported in case study 2 described below. 
Different course teams adopted different solutions, including different comput- 
ers, as there was no leading market standard or computer at the time. For exam- 
ple, one low population course found funds to buy computers in order to loan 
these to students. Student demand was also increasing: many students wanted. 
to use a computer for their OU study - or were already using one and wanted 
guidance on what to use or buy: 


“By 1984 the university was considering the feasibility of specifying one 
particular machine which would primarily serve the computer science 
courses, but would have the capacity to handle a variety of software appli- 
cations” (Jones et al., 1993, p.44). 
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In the end the direction taken was to define the equipment according to com- 
patible software, thus developing a home computing policy which specified an 
operating system. A core policy team oversaw the project, and a large team 
of academics and staff from the student computing services collaborated with 
senior university managers to conduct studies of students’ and tutors’ practices, 
to run pilot projects, and capture student and tutor experiences. Twenty differ- 
ent reports were written over the period of the project (1988 - 1991); report- 
ing on diverse aspects including the use of computer-mediated communication 
(Mason, 1988), computing on mathematics courses and tutor use of the home 
computing facility more generally (Kirkup and Dale, 1989). The pedagogical 
context of the university was particularly important given the OU’s commit- 
ment to openness and accessibility. The wider context of what computers were 
available at the time was clearly crucial - and a policy was needed that could 
respond to changes in the wider environment. That was achieved by defining 
software requirements rather than hardware. 

The development of the policy and its evaluation is a good example of the 
Beyond Prototypes model of the TEL complex in practice. The key features of 
the project that determined its success were: 


e Commitment to the policy at a senior level in the OU, underpinned by 
the importance of providing access to computers which, given student 
and course team demand, was argued as being crucial to the university’s 
core business. 

e The idea originated with academic staff and ‘spread upwards (Jones, Kirkup 
and Kirkwood, 1993, p.148) so had strong champions who prepared the 
ground and developed the argument. This and the previous point show the 
importance of context. 

e Key players included the chair of a very large population course, which 
needed access to a computer for the preferred design of the course to work. 

e The policy aimed to provide affordable and accessible access for our stu- 
dents in line with the OU mission: ‘to be open to people, places, methods 
and ideas’ and to ‘promote...educational opportunity and social justice. 

e Alongside a history of collecting evidence about our student learning lay a 
commitment to evidence-based research: so the evaluation findings had a 
ready audience in appropriate university committees. 

e The evaluation took a broad approach. Issues highlighted by the evaluations 
included the students’ social and physical context; issues of access and equal 
opportunities; teaching practical computing work at a distance; the design 
of learning materials and institutional support. 

The OU saw itself, and was viewed externally, as innovative. 


One chapter in Jones et al.(1993) devotes itself to an analysis of why and how 
the university adopted the HCP. This adds an additional dimension to the fea- 
tures above, which is the political and economic context of the 1980s when a 
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period of recessions and contraction began. There were a number of challenges 
to the institution, and the OU felt particularly under threat, and needed to find 
ways to cut production costs. The policy had powerful and persuasive champi- 
ons who carefully laid the ground and made preparations for the final debate 
at the university’s senate. The chapter concludes (see Jones et al. op. cit., p.148) 
that “one of the most interesting aspects of the HCP was that it was an idea 
that spread upwards from the academic staff who argued it through formal and 
informal channels in such a way that ownership of the idea became diffused 
throughout the institution.” 


Case study 2: Works Metallurgist (1996) 

“The Works Metallurgist”, (Blake et al., 1996), was an interactive tutorial 
designed to teach interpretations of phase diagrams and the Lever Rule (a 
method of calculating percentage of solid and liquid in an alloy at a given tem- 
perature from a phase diagram) to Open University students. It was developed 
for “Materials: Engineering and Science’, a second-level course which ran from 
February until October and included a residential summer school. Course eval- 
uations had shown that students had difficulties in interpreting and applying 
phase diagrams, and the program was specifically developed to help students 
in this area. It was in a game format, and students were given job titles ranging 
from Applicant for Apprentice Metallurgist to Works Metallurgist, according 
to their performance. 

Students used the CAL programs during their laboratory work. The program 
had been designed for individual use, but students mainly worked in the labo- 
ratories in pairs, and usually chose to use the program in pairs too, and to 
discuss their answers with each other before typing them in. Three comput- 
ers were also provided in the student hall for use at any time. The aim was to 
answer the following questions: 


1. How do students use the program? 
2. Does the software contribute to learning? and if so what do students learn? 
3. How can it be improved? 


The participants were 540 students who studied “Materials: Engineering and 
Science” in 1995. The researcher attended two weeks of residential school (out 
of seven) and conducted observations and interviews. The students had reason- 
able familiarity with computers, and were given a questionnaire, attitude scale, 
and knowledge pre-test, along with the evaluation disk (a special version of the 
program that recorded some usage information). 

Forty-four sessions were observed; both in the laboratory and the student 
hall where the program was also available for use during summer school. 
Where appropriate the students were asked to supply reasons for their answers. 
The observer tried to minimize the disturbance to students’ natural progress 
with the program, but was occasionally asked for help with the tasks, and this 
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was used as an opportunity to ask probing questions. A two-stage question- 
naire was given to 60 randomly selected students, where stage 1 asked about 
previous OU courses, study of phase diagrams, and the students’ computing 
background. This provided an attitude measure and a knowledge pre-test. 

Stage 2 concerned the efficiency of the program, the quality and nature of 
students’ interactions with it, their difficulties, their opinions of the program, 
and general comments. These 60 students were given a special version of the 
program which recorded usage information and were asked to return the disk 
after five weeks. This evaluation disk enabled us to see how much time stu- 
dents spent on the program, which sections took most time, and how much 
improvement in their understanding they made in that limited time. To meas- 
ure their learning achievement students completed a knowledge pre-test along 
with the first part of the questionnaire, and they were tested again (using the 
same questions) after they had finished working with the program. Four more 
short questions covered the important concept of phase. Formal and infor- 
mal interviews were conducted in the laboratories and in the computer suite 
in the student hall. Tutorials related to phase diagrams were also attended 
by the researcher, and students’ attendance and activities during the tutorial 
were observed. 

Observations revealed that The Works Metallurgist was the program used 
most during the 2 summer-school weeks. It was available for sale and most 
students who saw it decided to buy it, if they had access to an appropriate com- 
puter. The observations about its use and popularity were supported by the 
sales figures. Students reported that the program was very useful, and most 
could only suggest making minor changes to it. None of the students reported 
the program as being difficult to use, and they were easily able to use specific 
features such as Crosshairs, and the Draw Tie Line and Show Labels facilities. 
Evaluation disks returned showed that they spent a great deal of time using the 
program, ranging from 12 to 276 minutes with a mean of 128 minutes. 

The pre-post test data showed little difference in their knowledge before and 
after using the program. At that time, we noted that the difficulties in using 
pre-and post-tests in CAL evaluation were known and documented and that 
such instruments are not sensitive to the complexity of the learning situation. 
Data from the evaluation disks showed that the errors made in each section 
decreased with time. The students also commented that the game-like nature 
of the program was motivating. We also had data from 29 students who were 
positive about having learnt from using the program and in particular felt that 
it had helped them in understanding phase diagrams. 


Reflections on case study 2 


As in case study 1 we did have usage data for the students that used the eval- 
uation version of the program (but at a very small scale - 60 students). In 
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addition to the questionnaires and interview used in this study as with case 
study 1, students were observed using the program and students also com- 
pleted knowledge tests, although the pre-post test data showed little difference 
in their knowledge before and after using the program - a common result in 
CAL evaluation. 

The main link between this work and learning analytics is that both are con- 
cerned with understanding student behaviour and use and improving teaching 
as a result — but that in our historical CAL evaluations we were focusing on one 
aspect of a course - the CAL component. 


Summary and Conclusions 


In this chapter we have described some very early work of the CAL research 
group, noting the influences on work in the late 1970s and in the ‘80s from 
research in cognitive science and Intelligent Teaching Systems. Thus the meth- 
ods adopted for such work often focused on observations of student learning 
and collecting detailed data of students interacting with written texts of CAL 
or of whole models. Earlier studies often used protocol data in the laboratory: 
in later work, the CALRG ‘data capture suite’ provided video data of students 
working collaboratively. The commitment to a fine-grained understanding of 
student behaviour, use and learning with technology and improving teach- 
ing as a result is echoed in today’s learning analytics work (see Chapter 7) 
although of course this focuses on the student experience throughout the 
course or module. 

Major goals of our early work were to develop a better understanding of stu- 
dent behaviour, and to improve instructional design. Also, when the course 
materials consisted of texts that lasted a number of years, it allowed feedback 
to inform changes to CAL: this could be changed in a way that print could not. 
Some elements have persisted through time. One such element, the aspiration 
to have built in evaluation and data collection, was successful, although this 
was much more limited in the 1970s and early 1980s. 

CAL evaluation: institutional research, was illustrated by two case studies of 
CICERO (tutorial CAL) and the Metallurgy works. One of the findings in the 
CICERO evaluation was that affective issues were important — as well of course 
as the importance of integrating any CAL closely with the course and providing 
good accessibility. 

So, two main strands of work have been identified here; firstly a technology- 
focused approach influenced by AI and focusing on learner models, and sec- 
ondly a focus on research that investigated what learners’ needs actually are. In 
both these approaches, the OU was ahead of its time. We are now witnessing a 
ressurgence of interest in applying AI to education (see, e.g. Luckin and Hol- 
mes, 2016) but in 1978 it was unusual to teach AI as part of cognitive psychol- 
ogy. The university was also breaking new ground in researching into its own 
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students: in collecting and analysing usage data of student CAL use, in try- 
ing out mixed-methods methodologies, exploring methods of data capture, in 
detailed analysis of interactions and in considering affect. Some of this research 
laid the ground for the current work on learning design and analytics which is 
described in the next chapter. 

Some of the lessons learnt from this early work for meeting learners’ needs 
are still valid and really important today: 


e It is not possible to meet learners’ needs without an understanding of peda- 
gogy and how people learn; 

e Context is vitally important and needs to be considered in different ways; 

e Affect is very important: emotions such as fear and embarrassment have a 
significant effect on learning behaviours, yet it is only during the last fifteen 
years or so that this has been widely acknowledged and become part of 
mainstream educational technology research; 

e Assessing how people learn and how they learn best is challenging because 
of the nature of the learning situation. 


The chapter has also considered a successful institutional innovation - the OU’s 
Home Computing Policy - that was analysed at the time in terms of the eco- 
nomic and political climate, both at a local and wider level. However, the HCP 
is also a good illustration of the TEL complex in practice. One of the important 
components of the policy’s access that was listed was the commitment to the 
policy at senior university level. This is not a novel argument or finding, but 
perhaps the fact that it was also ‘bottom-up’ and so had ownership amongst 
academic staff and our associate lecturers is significant. 
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CHAPTER 7 


Evidence-Based Learning: Futures 


Using learning design and learning analytics to 
empower teachers to meet students’ diverse needs 


Bart Rienties and Ann Jones 


With the introduction of learning design in early 2000 and learning analytics in 
2012, the OU has led the way in how teachers make complex decisions to design 
interactive courses, and how students can maximise their learning potential. The 
next obvious steps would be to include AI, personalisation, and student-led learn- 
ing analytics to provide learning opportunities that meet the unique needs of each 
learner, but whether this would be technically feasible and pedagogically desir- 
able will be discussed. In this chapter we will look at recent and future develop- 
ments concerning the “holy trinity” of learning design, learning analytics, and 
how teachers can help institutions like the OU to ensure that our current and 
future students’ needs are met. Furthermore, we will reflect on the affordances 
and limitations of learning design and learning analytics to help teachers to adapt 
their teaching and learning practices to meet learners’ needs. 


Introduction 


The Open University (OU) has been at the forefront of innovation in teaching 
and learning since its inception in 1969. As highlighted in the previous chapter, 


How to cite this book chapter: 

Rienties, B. and Jones, A. 2019. Evidence-Based Learning: Futures. In: Ferguson, R., 
Jones, A. and Scanlon, E. (eds). Educational Visions: Lessons from 40 years of inno- 
vation. Pp. 109-125. London: Ubiquity Press. DOI: https://doi.org/10.5334/bcg.g. 
License: CC-BY 4.0 


110 Educational Visions 


even when most people did not have access to a computer, let alone a smart 
phone, the OU was actively experimenting and rolling out innovative ICT sys- 
tems and applications to help support teachers to deliver exciting and relevant 
approaches to help students meet their needs. In this chapter, we will primarily 
reflect on major developments of innovative teaching practice since 2010 that 
have shaped the OU and wider environment, and vice versa. In particular, this 
chapter will focus on the holy trinity between learning design, learning analyt- 
ics, and teachers. 

Like many other institutions across the globe, as highlighted in Chapter 6 the 
OU continuously explores the opportunities information technology affords 
to provide a better, more consistent, and ideally more personalised service to 
its learners, teachers, and wider stakeholders (Herodotou et al., 2017; Hidalgo, 
2018; Rienties & Toetenel, 2016; Tait, 2018). Globally (Dalziel, 2016; Hernan- 
dez-Leo et al., 2018; Lockyer & Dawson, 2012; Mangaroska & Giannakos, 2018) 
as well as within the OU (Nguyen et al., 2017; Rienties et al., 2018a; Rienties & 
Toetenel, 2016; Toetenel & Rienties, 2016a; van Ameijde et al., 2018) there is 
an increased recognition that learning design is an essential driver for learning, 
as well as empowering teachers to meet students’ needs. For example, using 
concepts originally developed by Conole (2012) the Open University Learn- 
ing Design Initiative (OULDI) has been implemented on a large-scale within 
the OU (Cross et al., 2012; van Ameijde et al., 2018). An excellent example of 
this large-scale implementation comes from a review of 157 learning designs of 
OU modules, whereby Toetenel and Rienties (2016a) found a wide tapestry of 
interactive and unique learning designs, from more traditional constructivist 
designs to more socio-constructivist designs. 

However, learning design by itself is just a useful approach to depict how 
teachers design a particular learning activity or a complete course. Only when 
learning design is combined with how students are actually engaging with these 
learning designs do we start to make real progress. One way to empower learn- 
ing design is to use learning analytics data of students. As argued by a range 
of researcher and practitioners (Calvert, 2014; Ferguson et al., 2016; Hlosta et 
al., 2015; Toetenel & Rienties, 2016a; Wolff et al., 2013) learning analytics may 
empower distance learning institutions like the OU to provide near real-time 
actionable feedback to students and teachers about what the “best” next step in 
their learning journeys might be. For example, the OU uses learning analytics 
dashboards displaying learner and learning behaviour to our academic staff 
and associate lecturers (ALs) in order to provide more real-time, or just-in- 
time support for students. (Herodotou et al., 2017, 2019; Hlosta et al., 2015). 
Furthermore, some institutions like Universiteit van Amsterdam (Berg et al., 
2016), University of Keele (de Quincey et al., 2019), and Maastricht Univer- 
sity (Tempelaar et al., 2018b) have successfully experimented with providing 
learning analytics data directly to students in order to support their learning 
processes and self-regulation. 
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As also highlighted in Chapter 3, the role of teachers in making sense of 
these dynamic and complex systems is vital. In fact, how teachers are making 
sense of the teaching and learning practice, its students, and data arising from 
the complex interactions of students with learning resources, peers, and teach- 
ers, has become even more important in the last 5-10 years (Herodotou et al., 
2017, 2019; Hidalgo, 2018; Rienties et al., 2016a; 2018a, 2019; Tait, 2018). As 
demonstrated by a range of projects within the OU as well as outside the OU 
(Guri-Rosenblit, 2018; Lawless & Pellegrino, 2007), the teacher is the key suc- 
cess factor in making pedagogy and technology work. As highlighted elsewhere 
in this book in Chapter 2, the OU has a relatively unique approach to teach- 
ing and learning, whereby typically central academic staff supported by TEL 
professionals design and produce high-quality online courses (Jones & Issroff, 
2005; Jones et al., 1996). The actual implementation and “teaching” of these 
modules (i.e., courses) is done by a combined team of module academics and 
ALs, who typically would support around 20 students per group (Herodotou 
et al., 2017; Toetenel & Rienties, 2016a; van Ameijde et al., 2018). With current 
movements towards co-creation and integration of ALs and (former) students 
into module production and presentation, in this chapter we use the broader 
notion of a “teacher” to refer to a person working together with other experts to 
effectively design, implement, and/or evaluate the teaching and learning prac- 
tices to meet students’ needs (Olney et al., 2018; Rienties et al., 2013, 2019). 

Using the Beyond Prototypes framework developed by Scanlon et al. (2013), 
which is described in Chapters 1 and 2, we will aim to illustrate how the holy 
trinity of learning design, learning analytics, and teachers can help institutions 
like the OU to ensure that our current and future students’ needs are met. The 
Beyond Prototypes case studies that led to the development of the framework 
indicate that Technology Enhanced Learning (TEL) needs to be understood 
as a ‘complex’ This ‘complex’ is made up of a series of elements that need to be 
addressed together, as reproduced in Figure 1.1 of Chapter 1 of this book. 

The second to outer level of Figure 1.1 shows the different communities 
that are all involved in the TEL complex: the student community, pedagogic 
research community, teacher community and technical communities. These 
communities are all necessarily involved in our learning analytics work (Fer- 
guson et al., 2016; Herodotou et al., 2019; Rienties et al., 2019). This work is 
being undertaken by a group of researchers within the Computers and Learn- 
ing research group (CALRG). One aspect of that work is what we call data 
wrangling’ (Ullmann et al., 2018) and includes iterative conversations with 
academics in the university who are responsible for developing our modules. 
Essentially the data wranglers team interpret the student data and then have 
conversations with academics about how changes might be made to the mod- 
ules to improve student learning. 

As illustrated in Figure 7.1, there may be inherent tensions between the three 
base layers of learning analytics, learning design, and teachers. Depending on 
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Students’ needs 


Learning:Design 


Learning Analytics Teachers 
Figure 7.1: Balancing learning design, learning analytics, and support by 
teachers to meet students’ needs. 


how effectively organisations are able to balance these three “forces”, the more 
we can meet the unique and individualised students’ needs (i.e., the higher or 
flatter the pyramid will become). 


Learning Design 


As highlighted by a systematic review of 43 studies on learning design by Man- 
garoska and Giannakos (2018) few institutions have implemented learning 
design on such a large scale as the OU. Conole (2012) started experimenting 
with mapping learning design processes, whereby they “developed an approach 
to using learning design as a methodology to guide design and foster creativity 
in concert with good practice in the creation of learning activities” Building 
on this initial work, the OU’s learning design taxonomy was established as a 
result of the Jisc-sponsored OU Learning Design Initiative (OULDI) (Cross et 
al., 2012; Rienties et al., 2017), and was developed over five years in consulta- 
tion with eight other Higher Education institutions. In contrast to instructional 
design, learning design is process based (Conole, 2012): following a collabora- 
tive design approach in which OU module teams, curriculum managers and 
other stakeholders make informed design decisions with a pedagogical focus, 
by using representations in order to builda shared vision. For a detailed descrip- 
tion of the OULDI approach, we refer to work published elsewhere (Rienties et 
al., 2017; Rienties & Toetenel, 2016; van Ameijde et al., 2018). 

In one of the first studies to visualise the complex decisions that OU teach- 
ers make when designing courses, Toetenel and Rienties (2016a) used the 
OULDI approach to classify 157 modules at the OU. As illustrated in Figure 7.2, 
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substantial depth and breadth of learning designs is present at the OU, perhaps 
reflecting the unique and diverse nature of the disciplines and the creative peo- 
ple that work at the OU. A considerable number of OU modules had a relatively 
high focus on assimilative activities, as well as assessment. At the same time, 
some OU modules used a perhaps more innovative pedagogical design, whereby 
for example Module 94 had nearly 60% of productive activities (ie., creating, 
building, making, doing) for students to work with, while nearly 40% of activi- 
ties in Module 56 were experiential (i.e., practice, apply, mimic, experience). 
Perhaps surprisingly for an online distance institution, less than 5% of learning 
activities on an average of modules mapped in 2016 were devoted towards com- 
munication activities (i.e., student to student, staff to student, student to staff). 


Assimilative Findi Communication Productive Experiential interactive Assessment 
information 


Figure 7.2: Learning design across 157 modules at the OU (activities in %). 
Retrieved from Toetenel and Rienties (2016a). 
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Figure 7.3: Changing OU teachers’ learning design (before and after visualisations). 
Retrieved from Toetenel and Rienties (2016b). 
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In follow-up work of 148 learning designs by Toetenel and Rienties (2016b), 
the introduction of a systematic learning design initiative, consisting of visu- 
alisation of initial learning design and workshops, helped OU teachers to focus 
on the development of a range of skills and more “balanced” learning designs. 
As illustrated in Figure 7.3, when OU teachers were given visualisations of their 
initial learning design activities (i.e., orange) compared to teachers who were 
not given these visualisation (ie., blue), they adjusted their designs towards 
more student-active activities, such as communication and finding informa- 
tion, while reducing the emphasis on assimilative activities. 


Learning analytics 


Although these above visualisations of learning design decisions made by 
teachers are an important advancement in terms of understanding our design 
practice, a next logical step would be to explore how these learning design deci- 
sions influence students’ affect, behaviour, and cognition. One way to do this 
is to use learning analytics, which is commonly defined as “the measurement, 
collection, analysis and reporting of data about learners and their contexts, for 
purposes of understanding and optimizing learning and the environments in 
which it occurs” (Ferguson, 2012, p.307). As noted by Scanlon et. al. (2013, 
p.37) “learning analytics can provide actionable intelligence”. 

A considerable literature from the OU has emerged around both conceptual 
development (Clow, 2013; Ferguson, 2012; Ferguson & Buckingham Shum, 
2012), how to evidence that learning analytics works (Ferguson et al., 2016; 
Ferguson & Clow, 2017; Rienties et al., 2016b), and how to design appropriate 
predictive learning analytics to effectively support different groups of OU stu- 
dents (Calvert, 2014; Herodotou et al., 2017; Rienties et al., 2019; Wolff et al., 
2013). In fact, a recent bibliometric review of learning analytics has found that 
the OU is the most prolific institution in publishing about learning analytics 
(Adeniji, 2019). 

With the arrival of fine-grained log-data and the emergence of learning ana- 
lytics as a research field there are potentially more, and perhaps new, opportu- 
nities to map how students with different affective, behavioural, and cognitive 
learning needs want to engage with the OU (Nguyen et al., 2018a; Rienties et al., 
2019; Rogaten et al., 2019). This is part of a commitment to investigate students’ 
practices: part of our efforts to understand our learning ecology, an impor- 
tant layer of the TEL complex captured in Figure 1.1 in Chapter 1. As noted in 
Chapter 6, the CALRG’s early work included a focus on affect. This was unusual 
at the time but this emphasis has continued. For example, with trace data on 
students’ affect, the OU is currently exploring how emotional expression could 
be identified in written text, such as chat, discussion forums, or feedback from 
students (Aznar et al, 2016; Chua et al., 2017; Hillaire et al., Submitted; Ull- 
mann et al., 2018). For example, Hillaire et al. (Submitted) showed that effective 
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sentiment analyses approaches could be developed to identify positive, nega- 
tive, and mixed emotions when 500+ students collaborated online in an inter- 
active chat environment. Similarly, Ullmann et al. (2018) found, when using 
sentiment analyses of 51,000 student evaluation comments from 23 large OU 
modules, that substantial differences in lived and affective experiences could 
be identified. 

Currently, experiments using techniques like eye gaze investigate how stu- 
dents are making sense of complex and simple texts (Rets, 2018). Furthermore, 
a range of studies within the OU have combined self-reported dispositions with 
how students are engaging with tasks over time (Tempelaar et al., 2012, 2015, 
2018a). These affective data could be useful in providing more personalised 
feedback to students, such as giving automated hints to a “surface” learner with 
math anxiety that, say, engaging with a worked example on task 15 would help 
him to better understand this math problem and reduce his math anxiety, while 
for a “deep” but disengaged learner for the same task 15 providing a hint to read 
the theoretical modelling narrative could prevent her from being bored. 

In terms of students’ behaviour, substantial progress has been made over the 
last five years in terms of identifying and predicting effective behaviour (e.g., 
engagement, time on task, clicks). For example, our state-of-the-art predictive 
learning analytics system called OU Analyse has been providing effective sup- 
port to hundreds of teachers across dozens of modules where students might 
need some additional support (Herodotou et al., 2017; Hlosta et al., 2015; Wolff 
et al., 2013). OU Analyse uses a combination of machine learning and artifi- 
cial intelligence approaches to predict which students are doing well, and who 
might be at risk not submitting the next assignment. One remaining challenge 
for learning analytics research is to deliver “actionable feedback’, which might 
be achieved by taking into account the context in which learners, teachers, and 
the respective learning data is situated (Chua et al., 2017; Herodotou et al., 
2017; Hidalgo, 2018; Rienties & Toetenel, 2016). 

Finally, in terms of students’ cognition some substantial progress has been 
made in the OU to signpost students about what they could do next, and what 
might fit better with their learning needs. For example, several module teams 
have been experimenting with asking for real-time feedback from students. 
Similarly, several module teams have implemented Computer-Based Assess- 
ments (CBA), which give automatic feedback to students (Nguyen et al., 2017). 
Preliminary analyses across 74 modules seemed to indicate that these CBA have 
a positive impact on engagement of students, and on higher pass and retention 
rates (Nguyen et al., 2017). 

At the same time, as argued by Rienties et al. (2019) in a recent review held 
during an interactive workshop of leading experts and users of learning analytics 
at the OU, many of the 42 participants indicated a strong need to further develop 
learning analytics approaches to allow for effective communication and person- 
alisation with students, while at the same time providing the learning analytics 
tools as part of an integrated design that is based upon a solid evidence-base. 


116 Educational Visions 


Linking learning analytics with learning design 


In terms of linking learning design with learning analytics approaches, several 
substantial steps have been made by CALRG researchers in the last five years 
(Mangaroska & Giannakos, 2018; Rienties et al., 2017). For example, Rienties 
and Toetenel (2016) linked 151 modules taught in 2012-2015 at the OU fol- 
lowed by 111,256 students with students’ behaviour using multiple regression 
models and found that learning designs strongly predicted Virtual Learning 
Environment (VLE) behaviour and performance of students, as illustrated in 
Figure 7.4. Findings indicated that the primary predictor of academic retention 
was how teachers designed their modules, in particular the relative amount of 
so-called “communication activities” (i.e., student to student, teacher to stu- 
dent, student to teacher). 

In contrast, student satisfaction was negatively predicted by these communi- 
cation activities, whereby students in particular preferred to work in modules 
following more traditional distance learning designs, such as constructivist 
learning designs. This may be an important finding as in particular in online 
learning there tends to be a focus on designing for individual cognition rather 
than social learning activities (Arbaugh, 2014; Koedinger et al., 2013), while 
recently several researchers have encouraged teachers and researchers to focus 
on the social elements of learning (Arbaugh, 2014; Ferguson & Buckingham 
Shum, 2012) 

Building on this initial work, Quan Nguyen has made substantial steps 
towards more dynamic, temporal conceptualisations and empirical analyses 
linking learning design from a day-week-module perspective with how stu- 
dents are actually engaging (Nguyen et al., 2017, 2018a, 2018b) For exam- 
ple, a large-scale empirical study by Nguyen et al. (2017) on learning designs 
of 74 modules over 30 weeks revealed that the way teachers designed their 
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Figure 7.4: Learning design strongly influences student behaviour, satisfaction 
and performance (Adjusted from Rienties and Toetenel (2016). 
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Figure 7.5: Longitudinal visualisation of learning design and student engagement. 
Retrieved from Nguyen et al. (2017). 


learning designs could explain up to 69% of the variance in VLE behaviours. 
For example, the weekly workload of the seven learning design activities of 
one module is illustrated in Figure 7.5. As highlighted from this visualisation, 
there were substantial fluctuations in expected workloads on a weekly basis, 
whereby there were specific weeks with a relatively high workload, primarily 
links to assessment points. As indicated by the red line in Figure 7.5, the aver- 
age VLE engagement of students on a weekly basis also fluctuated, and primar- 
ily peaked when assessments were due. In follow-up work looking at when 
and what students are engaging with in one fully online module, Nguyen et 
al. (2018a, 2018b); found that students made conscious decisions not to follow 
the course schedule, by either studying well in advance, or catching up after 
the course schedule. 


Role of teachers in using learning analytics and learning design 


Irrespective of the specific learning design and the learning analytics 
approaches used, teachers will always play an essential role in online and dis- 
tance learning (Guri-Rosenblit, 2018; Lawless & Pellegrino, 2007; van Leeu- 
wen et al., 2015). Several authors (Herodotou et al., 2017; Rienties et al., 2013, 
2018a) have indicated that beyond designing learning activities and managing 
the learning process teachers have a social, personal counselling role, whereby 
teachers provide pedagogical support and evaluate learning progression and 
outcomes. With the advancements of learning design and learning analytics it 
is anticipated that teachers will increasingly receive unprecedented amounts of 
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information, insight, and knowledge about their learners and their diverging 
needs. Learning analytics dashboards may provide teachers with opportuni- 
ties to support learner progression, and perhaps personalised, rich learning 
(FitzGerald et al., 2018; Rienties et al., 2016b; Tempelaar et al., 2015). Indeed, 
two recent systematic reviews of 26 and 55 learning analytics dashboards 
studies (Jivet et al., 2018; Schwendimann et al., 2017) indicated that teachers 
and students will be able to obtain (almost) real-time information about how, 
where, and when to study. 

Beyond providing just-in-time support (Daley et al., 2016; Herodotou et al., 
2017), learning analytics may help teachers to fine-tune the learning design if 
large numbers of students are struggling with the same task (Hidalgo, 2018; 
Rienties et al., 2016a; Rienties & Toetenel, 2016). In line with the Beyond Pro- 
totypes framework, paying attention to the ecology of practices, one layer of the 
policy context is a key element of our approach to learning analytics, so teacher 
and learning practices and perceptions are investigated. Regarding teachers, 
a recent large-scale study by Rienties et al. (2018a) amongst 95 experienced 
teaching staff at the OU indicated that many teachers were sceptical about 
the perceived ease of use of learning analytics tools. Most teachers indicated a 
need for additional training and follow-up support for working with learning 
analytics tools. 

These findings resonate with a recent study by Herodotou et al. (2017), who 
compared how 240 teachers made use of learning analytics predictions and 
visualisations in OU Analyse at the OU (Hlosta et al., 2015; Wolff et al., 2013). 
Herodotou et al. (2017) found that most teachers struggled to turn learning 
analytics predictions and recommendations into concrete actions for their stu- 
dents-at-risk. Follow-up qualitative interviews with five teachers who used OU 
Analyse indicated that they preferred to learn a new learning analytics system 
by experimenting and testing the various functionalities of learning analytics 
dashboards by trial-and-error (Herodotou et al., 2017; Herodotou et al., 2019). 
However, at this moment the OU does not actively track how teachers are mak- 
ing interventions, and what the best way could be to provide effective feedback 
for different groups of students. 


Conclusion and future directions 


In the last ten years universities and distance learning institutions like the OU 
have experienced unprecedented change. Beyond the “neo-liberalist waves” 
running through many universities, the affordances and limitations of technol- 
ogy to transform universities as exciting and relevant places of learning and 
teaching have fundamentally impacted the way universities are run, as indi- 
cated elsewhere in this book in Chapter 2 and Chapter 3. 

The central message of this chapter is that learning design, learning analyt- 
ics, and teachers together can support student success. With the emergence 
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of learning design combined with learning analytics, there is an increased 
narrative developing that teachers should start to pro-actively think, reflect, 
and act upon data. While there is widespread evidence that learning analytics 
tools and predictive engines could accurately identify which students might 
need some additional support, there is mixed evidence (Ferguson et al., 2016; 
Ferguson & Clow, 2017; Herodotou et al., 2017; Rienties et al., 2018a) as to 
whether universities and teachers in particular are ready to engage with these 
tools and approaches. 

As emphasised in the Beyond Prototypes framework: “In the TEL complex, 
practices include explicit aspects of teachers’ practices...” (Scanlon, 2013, 
p.29), and it is increasingly evident that without an appropriate understand- 
ing of the context in which learners and teachers are learning, learning ana- 
lytics may not be as effective as hoped (Ferguson et al., 2016). At the same 
time, our ground-breaking research (Nguyen et al., 2017, 2018a; Rienties et al., 
2018b; Rienties & Toetenel, 2016) linking learning design (i-e., what do teach- 
ers design) and what, how, and when students are actually engaging with these 
learning activities could have a transformative impact on how we teach at the 
OU, and perhaps more importantly how we can develop, test, and implement 
new educational theories of effective learning design. Given the tremendous 
impact of learning design on what students do on a daily and weekly basis 
(Nguyen et al., 2017, 2018b;), we need a much better understanding of why 
teachers are designing particular learning activities, and how these learning 
activities relate to learners’ needs. 

Indeed recent research has highlighted that on a more micro-level learners at 
the OU have substantially different learning needs and ambitions (Law, 2015; 
Li et al., 2017), depending on a complex interplay of affective (Hillaire et al., 
Submitted; Tempelaar et al., 2018b), behavioural (Chua et al., 2017; Rets, 2018; 
Rizvi et al., 2018), and cognitive factors, as well as socio-economic and demo- 
graphic factors (Richardson, 2015). Therefore, in the remainder of this chapter 
we will primarily focus on how to provide more personalised and individual- 
ised support for learning in the next 2-5 years. 


Moving forwards 


With the renewed and increased interest in Artificial Intelligence (Holmes et 
al., 2019; Luckin et al., 2016; Rizvi et al., 2018) there is an emerging narrative 
developing that universities should start to embrace some of the affordances of 
AI. In particular for some of the more mundane tasks that students and staff 
need to complete on a frequent basis (e.g., registering for a course, asking for an 
exception, replying to standard emails), providing automated responses using 
AI could provide some quick efficiency savings. Similarly, in providing auto- 
matic responses to standard or frequently asked questions, chat bots can learn 
to effectively support learners. 
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Personalisation (FitzGerald et al., 2018) and student-led analytics (Ferguson 
et al., 2017; Prinsloo & Slade, 2017) are two specific themes that are emerging 
from the literature that could start to play an important role for distance learn- 
ing providers in the near future. Although distance learning theoretically can 
provide flexible options to learners depending on their needs, we continue to 
see many distance learning providers offering “one-size-fits-all” courses start- 
ing on say the 1“ of October and finishing in June/July. Given that many learn- 
ers do not necessarily want to follow courses on these dates, some may want to 
start earlier or later, and others might want to move faster or slower (FitzGerald 
et al., 2018), it remains interesting why most providers of education are still 
focussed on one-size-fits all solutions. Obviously, economic efficiency argu- 
ments are provided, like economies of scale, and logistical and administrative 
processes need to be adjusted to accommodate multiple variations of a course, 
but with the support of learning analytics and a student-led analytics approach, 
distance learning organisations could vary their provision to different groups 
of learners, with specific learning needs. Finally, student-led analytics, whereby 
students themselves determine what they want to share and see in terms of 
their own data (Ferguson et al., 2017; Prinsloo & Slade, 2017), will become 
an emergent issue that distance learning institutions need to plan for. Again 
the Beyond Prototypes framework could be useful to help distance learning 
organisations to visualise the complex and changing relations. 
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CHAPTER 8 


STEM Learning: Foundations 


Eileen Scanlon, Christothea Herodotou, 
Denise Whitelock and Chris Edwards 


The first joint project undertaken by the Computers and Learning research group 
was the evaluation of The Open University Science Faculty’s CAL offering in 1979. 
Since then many CALRG activities such as PhD projects, major external research 
grants, and institutional contributions, have been directed towards a better under- 
standing of what makes science teaching and learning better. In this chapter we 
will consider our work on conceptual change in science and on the development 
of pedagogy and technology on personal inquiry using nQuire, and include work 
integrating these developments into the Open Science Laboratory. Our work has 
included evaluation of other innovative pedagogical supports such as the Puck- 
Land simulation for teaching Physics, Virtual Field Trips and the use of the Virtual 
Microscope both in the UK and a number of other UK and EU universities. We 
illustrate how judicious use of technology and pedagogy can promote enthusiastic 
engagement with science and give opportunities for participation and learning. 


Introduction 


At The Open University’s (OU) inception there were those who doubted that 
science can be taught at degree level to students accepted on an ‘open entry’ 
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basis, ie. without prior entrance qualifications. The OU developed science 
degrees from its first inception and the number of students registered with the 
university to start science degrees in the 1990s had reached ten percent of all 
the full-time science students in the UK (Pentz, 1978). These initial attempts 
were watched closely as there was some scepticism from the Higher Education 
establishment of the time that this could be done. 

Since the first attempts at teaching science in the OU fifty years ago, the suc- 
cessful learning experiences designed for Open University science students 
have demonstrated that so much is possible. The change in perceptions about 
the feasibility of learning and teaching science at a distance is in part due to the 
developments in the technologies available to support appropriate activities. 
The courses were designed to help students learning on their own so that every 
activity was designed carefully, following pedagogical principles, and tested 
before it was sent to students. This was important because OU students did not 
have a lecturer to explain things to them if anything was not clear. However, 
experiments could be refined and revised for residential schools. 

The importance of practical work in the teaching of science has always been 
an essential component of science teaching and emphasized in the literature (see 
e.g. Hofstein, 2007; Holstermann et al. 2010.) Practical work was dealt with by 
the incorporation of a variety of media in courses; including home experiment 
kits, radio or audiotapes, TV and laboratory classes at day schools as documented 
in Ross and Scanlon, 1995. Technology enhanced learning techniques and tools 
became more available and these were appropriate to help with this task too. 

Why is science hard to learn? There are particular challenges for teaching and 
learning science at a distance. The OU’s system offered the opportunity to study 
science to those with no previous qualifications in the subject. These ranged 
from the lack of prerequisites for study, the hierarchical nature of concepts 
needed to build science content knowledge, the need to develop mathematical 
skills, and develop practical work (Ross and Scanlon, 1995). 

Technology was deployed very early in the development of a pedagogy of 
teaching science at a distance at the OU. This included the introduction of 
technology to the mix of media adopted in the first years of the university. A 
media mix that was heavily text based included also broadcast TV, audio and 
from the mid-1970s the use of computers. In the late 1970s an evaluation of 
early attempts at the use of CAL in the science faculty reviewed remedial CAL 
tutorials delivered on terminals available (for limited access) at study centres 
and simulation programmes available at residential or day schools (see Scanlon 
et al. 1987, Jones et al. 1982. 1987a,1987b). In our first case study we consider 
the use of simulation and modelling arising from these first years’ experiences. 


Case Study one: Simulations and modelling 

Early experiments with the use of computers in teaching science involved sim- 
ulations and modelling. In a simulation the process or system is modelled and 
then made available to the user so that, by playing with the system, they can 
get some insight into what is being modelled. So, experiments can be simulated 
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that would be impossible to interact with in real life for reasons of access, 
expense time constraints or other considerations. A number of publications 
trace this work (e.g. Every and Scanlon, 1983; Ross and Scanlon, 1995, Blake & 
Scanlon, 1996, 2007). 

One issue which emerged from the evaluation of our CAL simulations (Jones, 
Scanlon, and O’Shea, 1987) was the importance of prior knowledge of the 
concepts which is often assumed in the design of the simulations. Sustained 
work on science learning in the past 40 years explored how science conceptions 
are developed. One example of our work on science concepts was the project, 
funded by the Economic and Social Research Council on using simulations for 
the development of conceptual change ( the ESRC-funded ‘Conceptual Change 
in Science’ project). Our aim was to promote change in learners’ understanding 
of physical phenomena. This work included the design and development of a 
set of teaching materials for the teaching of a topic on mechanics. These teach- 
ing materials included interactive computer simulations of force and motion, 
along with practical activities and written materials. The method involved mak- 
ing learners aware of the limitations of their current conceptions of force and 
motion. Then, learners would be helped to develop and use a conceptual frame- 
work which conformed to current scientific understanding. This would have to 
be in line with their experience and be internally consistent. Conceptual change 
was found to be developed in this sequence of lessons. The children on whom 
the curriculum was tested displayed more sophisticated reasoning than their 
counterparts in comparison classes (Hennessy et al. 1995a, 1995b). This work 
influenced the development of simulations used in distance education settings. 
One particular finding, that the conceptual change in science software was most 
effective in situations where practical experimental work was combined with 
work with simulations, was particularly important, see Twigger et al., 1994. 
(Another example of the influence of this work on the design of teaching, was 
the Supported Learning in Physics Project (SLIPP), where we used OU expertise 
in support of teacher education in schools Whitelegg and Edwards, 2001.) 

An extension of the simulation which consists of a mathematical model of 
some process underlying the system is that of a virtual environment. Laurillard 
(2001 p 36) points out ‘virtual environments use a graphical model to display 
the visual and positional properties of the system rather than its behaviour’. The 
examples of virtual field trips and virtual microscope are discussed below. 

Another connected strand of work involved a number of PhD theses on mod- 
elling physics problem solving, and graphical representations (link to CALRG 
theses lists in the library). These projects looked in detail at students’ current 
conceptions of different science topics and made use of detailed protocol anal- 
ysis of users’ interactions with computer systems as they solved problems to 
produce rich pictures (thick descriptions of students talking, writing and inter- 
acting with technology) to help us understand what ideas and interactions were 
causing difficulties with instruction (see also Driver and Scanlon, 1988). 

One particular problem that Physics students encounter is that of under- 
standing elastic collisions. A simulation known as PuckLand was developed 
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which allowed students to investigate the interaction between two ice pucks. 
This early work was written in HyperCard for use with the Apple Macintosh 
and consisted of a pair of pinball-style flippers on either side of the screen with 
which subjects could flick pucks (see Figure 8.1). 

The amount of force with which the flippers hit the pucks could be varied by 
raising the height of the flippers as could the mass of the pucks. When the ‘go’ 
button was activated the pucks moved towards each other on the screen and 
were animated with speeds proportional to those set by button presses. After 
the pucks collided, they moved away from each other with a speed that was 
calculated with the correct Physics formalisms. In this way, the principle of 
conservation of momentum and kinetic energy were obeyed as illustrated by 
the apparent screen velocities of the pucks. 

The empirical study undertaken with this simulation involved 16-17 year 
old Physics students working in pairs and this research was an early foray 
into computer supported collaborative learning. The initial findings revealed 
that students develop two families of causal models to explain motion after a 
collision. These have been identified as the linear causal and the resistance/ 
reciprocal causal model. However, these models broke down when the stu- 
dents were confronted with the animations from the PuckLand simulation. 


PuckLand 


Version 1.3 


| Before | After | 


Figure 8.1: Screen dump of the graphic interface to the PuckLand program. 
Reprinted from Computers & Education 20 (1). D. Whitelock, T. Taylor, T. 
O’Shea, E. Scanlon, R. Sellman, P. Clark, C. O‘Malley, Challenging models of 
elastic collisions with a computer simulation, pp. 1-9, copyright Pergamon 
Press Ltd (1993), with permission from Elsevier. 
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The result being that the students were moved to rethink their common- 
sense ideas about motion which lacked an understanding of conservation 
of energy. 

A second study involved pairing the students with either “similar” or “differ- 
ent” common-sense models of collisions. It is interesting to note this early work 
suggested that the cooperative construction of shared meaning rather than 
conflict was more important for successful collaboration (Barbieri & Light, 
1993). It is interesting to note that the PuckLand simulation, although routed 
in Science teaching and learning, was also able to prompt further investiga- 
tion into computer supported collaborative learning and to provide a further 
vignette into the practical application of social constructivist theory. 


Case study 2: Practical work-home kits, residential schools and multimedia 
approaches 

In an earlier Chapter (4) Coughlan et al. describe an approach to one particular 
problem of learning science at a distance, that of access to practical experiences. 
They describe one particular solution in the work researching remote laborato- 
ries (see also Scanlon et al., 2004). However, this was only one approach taken 
to the knotty problem of providing practical experiences for online distance 
learning of science. Initially the means of making practical experiences avail- 
able to students was the provision of extensive home experiment kits, residen- 
tial schools and broadcast TV programmes. 

More recently the provision of multimedia meant that students can have 
vicarious experience of observing experiments. In addition there is the pos- 
sibility of controlling variables and drawing inferences. An introductory sci- 
ence course offered at The Open University in the 1990s offered students the 
possibility of interacting with a global warming simulation, taking a virtual 
desert field trip and conducting Galapagos field trips (see e.g. Taylor et al. 
1996; Whitelock, 2001) sometimes with a problem-solving pedagogy applied 
(see e.g. Ross and Bolton, 1990). Virtual field trips where users explore a three- 
dimensional environment have been used (see e.g. Whitelock, 2001; White- 
lock and Jelfs, 2005). 

Furthermore, Whitelock with co-authors Brna and Holland (1996), selected 
three properties of virtual environments to incorporate into a model that could 
compare salient properties of virtual systems that would be open to test. These 
properties included representational fidelity, immediacy of control and presence 
which could define a finite but still a large space of VE classes. This model was 
used to understand the factors in virtual environments that promote concep- 
tual learning by comparing two desktop virtual environments which explored 
field trips to the North Atlantic Ridge by submarine and a walk through an oak 
wood (Whitelock, 1999). The representational fidelity was rated higher for the 
oak wood than the North Atlantic Ridge. Immediacy of control was perceived 
differently in the two VR programs due to the jerky movements experienced 
in these early desktop VR environments. However, the role of audio was found 
to be important in more than one way when virtual environments are being 
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built for conceptual learning. this is because they create a sense of presence, 
engagement and enjoyment but are also important for navigation. The latter 
is not such a trivial point as first may appear since the cognitive load could be 
diminished if navigation was easier and students could concentrate on under- 
standing and manipulating concepts within the subject domain. It is interest- 
ing to note that virtual reality systems still require navigational and conceptual 
compasses for conceptual learning as suggested from this 1999 study. Of a simi- 
lar vintage is our original Virtual Microscope project which simulates the views 
through a microscope of slides displaying different kinds of materials Whalley 
et al., 2011. This has particular benefits for students with accessibility issues as 
described in Chapter 4. For example, it provides access for students with dif- 
ficulty in reaching laboratories and better access to images for students who are 
partially sighted. 


Case study 3: Personal Inquiry project 
A significant advancement in the available technologies to support the devel- 
opment of science understanding had the effect of sparking a new extended 
investigation into how technologies can enable science learning in contem- 
porary contexts. In the context of a ‘Personal Inquiry’ (PI) project funded by 
the ESRC and Engineering & Physical Sciences Research Council (EPSRC), 
we explored the conditions under which evidence-based inquiry learning can 
be fostered. The PI project aimed to understand how personal and mobile 
technologies can be deployed to make the processes of evidence-based scien- 
tific inquiry readily accessible to young people (see Anastopoulou et al., 2008, 
2012). As Blumenfeld et al. (1991) point out, technology has a potentially use- 
ful role to play in structuring the process of inquiry learning with tactical and 
strategic support. 

Making scientific inquiry authentic is a challenge that has been discussed by 
a number of researchers (e.g. Edelson et al., 1999; Chinn and Malhotra, 2002). 
A personal inquiry toolkit was developed to scaffold this activity and was tried 
out in a number of school-based interventions, two in Nottingham schools and 
two in Milton Keynes. The young people carried out scientific explorations sup- 
ported by their teachers and also by a personal inquiry toolkit. This toolkit, in 
its first instantiation, ran on a small portable computer and guided the learners 
through a process of gathering and assessing evidence, whilst they conducted 
experiments on topic themes of relevance to the secondary-level UK National 
Curriculum. Further technology support was provided by data probes con- 
nected to the computer. Project partners included schools, technology compa- 
nies that develop sensing and data-logging equipment, museums, community 
resource centres and field trip sites. This broad partnership reflected our view 
that we need to support learning within the classroom and outside it whether 
on field trips or at home. 

At the culmination of this project we developed nQuire, a software applica- 
tion to guide personal inquiry learning. nQuire provides teacher support for 
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authoring, orchestrating and monitoring inquiries as well as student support 
for carrying out, configuring and reviewing inquiries. nQuire allows inquir- 
ies to be scripted and configured in various ways, so that personally relevant, 
rather than off-the-shelf inquiries, can be created and used by teachers and 
students. nQuire incorporates an approach to specifying learning flow that 
provides flexible access to current inquiry activities without precluding access 
to other activities for review and orientation. Dependencies between activities 
are automatically handled, ensuring decisions made by the student or teacher 
are propagated through the inquiry. nQuire can be used to support inquiry 
activities across individual, group and whole-class levels at different parts of the 
inquiry. It offers a flexible, web-based approach that can incorporate different 
devices (smartphone, netbook, PC) and does not rely on constant connectiv- 
ity (Mullholland et al., 2011). We published a set of studies of orchestration of 
inquiry within and beyond the classroom (Sharples, 2013; Sharples et al., 2015) 
and illustrated how the inquiry framework and nQuire toolkit together influ- 
enced the performance and effectiveness of inquiry learning (Littleton et al., 
2013). This paradigm has been further developed to scaffold online personal 
inquiry learning within informal settings. For example the nQuire platform has 
also been used in Higher Education in The Open University’s Open Science Lab 
(see Theme 2) to support informal examples of inquiry learning (Villasclaras 
Fernandez et al., 2013). 


Case Study 4: The iSpot Project 

The aim of iSpot (www.ispotnature.org) (see Figure 8.2) is to create a new gen- 
eration of naturalists by helping students and people of all ages learn how to 
identify organisms by enhancing natural history identification skills. It was 
launched in 2009 and developed initially with a five-year, £2 million grant from 
the Big Lottery Fund for England. This skill (to identify likely IDs for observed. 
flora and fauna) underlies all of biodiversity science. However, this is no longer 
widely taught in formal curricula in schools or universities. A South African 
site followed (www.ispot.org.za) and in 2013 a version for Chile was also cre- 
ated (www.ispotnature.org/chile). ispot has 65,000 registered users who have 
made more than 550,000 observations of many thousands of species. 

In working with the development of iSpot we have contributed to learning but 
also to scientific discoveries through observations communicated on the plat- 
form. The observations included two which had not been recorded in the UK 
before. A six-year-old girl discovered a moth on her windowsill. The moth, native 
to Asia, had never before been spotted in the UK. After identification on iSpot, 
the species was also confirmed by experts and the moth was taken into the Natu- 
ral History Museum collection. In addition, in South Africa, a doctor submit- 
ted a photograph of unknown seeds that were the cause of poisoning in several 
children presenting at a clinic and these were identified 35 seconds after posting 
on iSpot (iSpot 2013). Hitherto unknown populations of South African endemic 
plant species are regularly discovered on iSpot (Silvertown et al., 2015, p. 142). 
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Figure 8.2: The iSpot platform supporting species identification. 


iSpot was incorporated into the OU’s Open Science Laboratory which allows 
anyone, anywhere to access practical science education. iSpot also supports 
Open University / BBC broadcasts, the Open Science Laboratory, and was used 
in three environmental courses. 

We were involved in the design, development and running of the platform and 
the evaluation of learning on the platform (Scanlon et al., 2014). iSpot provides 
an online community where novices and experts can work together to identify 
living organisms and engage in crowdsourcing identification. iSpot allows any- 
one, anywhere, to upload an image for identification in towns, back gardens, 
open fields, forests and all sorts of habitats across the country. This contributes 
to solving the problem of learning about nature. Also, the information gathered 
by iSpot is used for other data collection for conservation purposes. 

The impact of iSpot on learning has been measured in a couple of ways. Learn- 
ing episodes are short and informal. So, we needed to think about the impact 
of iSpot and the potential outcomes, including increased awareness and impact 
on attitudes, as well as engagement and participation. It is complex to examine 
such learning settings as iSpot. Qualitative analysis does show clear examples 
of users who start as complete novices, and then come to a good understand- 
ing of identification. There is also some quantitative evidence of users learn- 
ing. For instance, analysis of a sample of 407 users as they progressed through 
submitting and identifying their first 50 observations within iSpot is strongly 
suggestive of learning as users showed improvement in their ability to identify 
other people’s observations over the period that they submitted observations: 
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As users progress from their first to their 50th observation posted on iSpot 
they have a bigger percentage of correct identifications, that is they are more 
likely to identify what they have seen for themselves. This change in behaviour 
probably reflects learning, although other causes of the trend may be possible 
(Silvertown et al., 2015). 


Conclusions 


A range of science-related projects have been undertaken over the last 40 years 
(see also the account in Scanlon 2011) which aims to support learners to learn 
science from a distance, and without previous academic qualifications, and 
to build connections between formal and informal learning in order to fos- 
ter their interest and curiosity in science. Major milestones in the journey of 
engaging learners with science have been the development of science simula- 
tions and modelling and their support of conceptual change (Case Study 1), 
technology mediated practical work, (Case Study 2), the Personal Inquiry (PI) 
project that scaffolded the process of scientific inquiry through technology 
(Case Study 3), and iSpot that leveraged the power of the crowd to support 
species identification (Case Study 4). This journey aimed to bring science 
closer to the everyday life of learners and help them understand and appreci- 
ate its value by developing bridges between formal and informal education. 
It also aimed to open up and make science accessible thus enabling poten- 
tially anyone to engage with science activities. These technological develop- 
ments promote enthusiastic engagement with science and give opportunities 
for participation and learning. For example, the use of simulations allows for 
hands-on experimental work to take place at any time, in a playful manner and 
by learning through failure, exploration and experimentation. Simulations 
lower the barriers to participation and make it easy for people to engage with 
activities often viewed as determined by scientists. In relation to our Beyond 
Prototypes themes (see Chapter one) this chapter illustrates the effect of the 
‘persistent intent’ of a succession of teachers and researchers who were deter- 
mined to meet our initial challenge from many commentators that as far as 
science learning at a distance goes, ‘It can’t be done’. 

The above line of work suggests that certain aspects can support the process 
of engagement with science including an understanding of what people need 
to know in order to effectively do science, both in formal and informal settings, 
such as knowledge of basic science-related concepts, and relevant mathemati- 
cal skills and skills for practical work. Also, an explicit account of hard to grasp 
concepts should be developed through studies with online learners, in order 
to identify and improve issues they are struggling with. Such an account could 
inform the design of more effective science learning experiences that consider 
the challenges or demands of self-regulated learning, such as the significance of 
appropriate scaffolding when learners complete tasks on their own. 
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CHAPTER 9 


STEM Learning: Futures 


Christothea Herodotou, Eileen Scanlon 
and Denise Whitelock 


Following on from the account of some CALRG research related to STEM learning 
in the previous chapter we discuss here several examples of attempts to explore the 
technology and pedagogy of learning science. In one thread we look at informal 
learning and linking journeys between formal and informal learning and how we 
have built on previous work on developing inquiry learning. In the second we look 
at a design to support collaborative working at a distance building on our previous 
work on learning from simulations. The case studies in this chapter illustrate the 
persistent intent of supporting science learners and a shared vision of the range of 
support under development for this end. 


Introduction 


Recent direction of this research has emphasized informal learning, journeys 
between formal and informal learning and collaborative working. The develop- 
ment of the nQuire-it project initiated a series of projects in the field of Citizen 
Science. It resulted in the development of the nQuire-it platform and the Sense- 
it app supporting the design and implementation of personally meaningful 
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investigations outside the classroom, by citizens of all ages. Through the BBC’s 
“Tomorrow’s World nQuire, this has developed further into a dynamic and 
social toolkit hosting multiple types of Citizen Science projects such as image, 
audio and text-based projects as well as survey-type projects with personal- 
ized feedback. In the LEARN CitSci project, our international Citizen Science 
collaboration with six natural history museums and universities in the US 
and UK is aiming to improve the design of existing Citizen Science projects 
led by museums and make science learning more enjoyable and accessible to 
young people. 


Case study 1: Citizen Inquiry 

The story of Citizen Inquiry is a strong example of how an innovative peda- 
gogy reinforced and guided the development of relevant technology. Pedagogy 
may often be overlooked in technology-enhanced learning (TEL) and this 
is due to the emphasis often placed upon technological elements rather the 
learning engagement, processes, and outcomes. In the case of Citizen Inquiry, 
advancements in technology enabled testing and evaluation of this pedagogy 
and supported progress towards a learning vision. Citizen Inquiry is an inno- 
vative approach to inquiry learning, proposed by OU Emeritus Professor Mike 
Sharples. It is located at the intersection between ‘Citizen Science’ and ‘inquiry- 
based learning’ and refers to mass participation of the public in joining and 
initiating inquiry-led scientific investigations. Specifically, “it fuses the creative 
knowledge building of inquiry learning with the mass collaborative participation 
exemplified by Citizen Science, changing the consumer relationship that most 
people have with research to one of active engagement” (Sharples et al., 2013, 
p 36). The ‘citizen inquiry’ paradigm shifts the emphasis of scientific inquiry 
from scientists to the general public, by having non-professionals (of any age 
and level of experience) determine their own research agenda and devise their 
own science investigations underpinned by a model of scientific inquiry. Citi- 
zen inquiry aims to leverage the pedagogical potential of inquiry-based learn- 
ing - a productive approach to the development of learners’ knowledge of the 
world and the enhancement of higher-order thinking skills - through opening 
up massive participation in inquiry-based activities. 

It becomes evident that Citizen Inquiry emerged from a reconfiguration of 
existing ideas and social practices, that is the increasing interest and growth of 
Citizen Science and its consequences for how science is conducted (Bonney et 
al., 2009) and Inquiry Learning as a problem-solving approach to learning that 
requires guidance and ‘scaffolding’ (Quintana et al., 2004). This is an example 
of ‘bricolage (Scanlon et al., 2013) where the central idea or the vision of TEL 
innovation resulted from examining available materials, approaches and ideas, 
and experimentation, rather than the configuration of a vision that is precedes 
the development of relevant material. A new research project may emerge from 
either inventing and testing a new idea, or through the process of bricolage, 
that is bringing together and reconfiguring what is already known. 
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The first step towards examining or testing the idea of Citizen Inquiry was 
to attract funding. In 2013, one-year funding from Nominet Trust enabled the 
recruitment of a software designer and an educational developer to design tools 
that could scaffold or facilitate learning through Citizen Inquiry. Advancements 
in technology such as synchronous and asynchronous communication, and the 
instant data upload from mobile phones to the web could enable communica- 
tion and mass participation of citizens in inquiry-led learning activities, and 
thus instantiate the idea of Citizen Inquiry. In partnership with teachers and 
students from the Sheffield University Technical College (UTC), we designed 
in a participatory manner the nQuire-it platform and a mobile data collec- 
tion application, Sense-it (available on Google play). The tools were developed 
through a design-based research approach, where prototypes of the tools were 
evaluated by young people, improved and retested (Herodotou, Aristeidou, 
Sharples, Scanlon, 2018, Herodotou et al., 2014). The delivery of functional 
versions of the tools was very timely as it coincided with the research activi- 
ties of a PhD student who was examining inquiry learning in online settings. 
In practice, this allowed for additional research to take place that could not be 
supported by the project funding, in two main areas: evaluation of the usability 
and general functionality of the tools and capturing early evidence of learning 
impact. It also extended the scope of tools to adult participants through a study 
with amateur meteorologists and weather watchers and provided recommen- 
dations as to how to sustain participation in citizen inquiry learning communi- 
ties (Aristeidou, Scanlon, Sharples, 2014). 

Despite the overall success of the project in developing and evaluating tools 
to support the process of Citizen Inquiry and a collection of papers published 
(Herodotou et al., 2017), a period of uncertainty followed the project comple- 
tion. Additional sources of funding were needed to sustain and monitor the 
tools, provide support to current and prospective users, develop and enhance 
the functionality of the tools and disseminate project outputs, and enable 
uptake of the innovation. During this critical period of no additional financial 
resources, the OU agreed to provide minimum technical support for main- 
taining the tools. Also, the research team made systematic attempts to attract 
additional funding. These efforts were directed both to external funders and 
the university; in the case of the latter, the intention was to integrate the tools 
and the new learning approach to the design of new or existing courses. What 
motivated these efforts was the fact that all involved researchers shared the 
same vision around the potential of Citizen Inquiry to make learning engaging 
and interactive while they were also willing to dedicate time to identify new 
sources of funding. In practice, we scheduled monthly or fortnightly meetings 
where we discussed our progress towards attracting funding and set actions to 
be completed before the next meeting. In this way, our efforts were systematic 
and persistent over time. 

Persistent intent (Scanlon et al., 2013) plays a significant role in pursuing and 
mainstreaming innovation in TEL. The sharing of a clear and common vision 
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amongst a research team can also help steer and plan future activities aiming at 
extending the project lifecycle. In the case of Citizen Inquiry, persistent intent 
resulted in a partnership with the BBC’s “Tomorrow’s World’ science program- 
ming (http://www.bbc.co.uk/tomorrowsworld), the aim of which was to extend 
existing tools and support large-scale public investigations in both the natural 
and social sciences (e.g., psychology). For example, tools should enable citizens 
to create and run online studies to explore attitudes and personality. This pro- 
ject, running for a year, involved certain technical developments including new 
ways to enter responses, secure handling of personal data, and ways for partici- 
pants to overview results. Overall, this funding enabled the re-implementation 
of tools to run large studies linked to BBC TV or radio programmes. By the end 
of the project funding, a new version of the now called nQuire platform had 
been developed (nQuire.org.uk) (see Figure 9.1). 

The platform was launched in 2018 with an investigation proposed by the 
BBC about the incoming General Data Protection Regulation: ‘“GDPR - My 
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Figure 9.1: The nQuire platform supporting citizen inquiry. 
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Life, My Data, MyTomorrow. Advertised on the BBC Tomorrow’s World 
website and systematically shared in social media, the expectation was that a 
significant number of users would visit the platform and complete the mission, 
thus raising the profile of the platform as a tool for joining or initiating investi- 
gations. A significant aspect of every piece of research is to achieve impact, in 
the case of Citizen Inquiry this means populating the platform with investiga- 
tions set by both citizens and scientists, supporting Citizens’ Inquiry learning 
through joining or setting up investigations, and integrating the tools in teach- 
ing practices. This first mission did not meet expectations, as only a few people 
participated. A reflective activity within the team identified that the process of 
accessing the platform, in particular the requirement to register before joining 
an investigation, most likely explained the low response rate. Given the remain- 
ing funding available, respective changes were made to the platform. 

The end of the project with the BBC resulted in a second critical point of 
uncertainty. Although this partnership had significant outputs, the challenge of 
achieving impact by having a large number of citizens systematically engaged 
with the tools had not yet been met. In line with ‘persistent intent, the research 
team shared responsibilities and directed dedicated effort towards two direc- 
tions: (a) identifying external organisations such as the BBC and schools that 
would be interested in using the platform and setting up investigations, and 
(b) identifying courses at the OU that could integrate the platform into their 
design to support teaching and learning. In this respect, networking was shown 
to be a critical factor influencing the future of the innovation. Through exist- 
ing contacts and ongoing communication, the research team identified rele- 
vant stakeholders to whom they presented the vision and the tools. This was 
a fruitful activity as it resulted in a number of outcomes. First, the Chrono- 
type mission was launched and shared with undergraduate students across 
the university, generating 4,700 responses. Second, leveraging internal funds, 
a researcher was recruited to identify ways of integrating the platform within 
Open University courses. Third, the platform was integrated into the design of 
a new course about technology-enhanced learning, that was managed by col- 
leagues in the same department. This is the first application of the approach and 
the tools to formal education. Fourth, the partnership with the BBC resulted in 
the design of additional investigations such as Gardenwatch (See Figure 9.2)- 
a partnership between the BBC and the British Trust for Ornithology (BTO) 
surveying gardens across the UK - that engaged thousands of people with the 
platform and the process of Citizen Inquiry. Finally, relevant applications are 
under preparation or have been submitted, aiming to attract further funding. 

In parallel, the dissemination activities of the research team over time, 
including presenting at local and international conferences and other universi- 
ties, attracted the interest of schools. For example, a primary school in China 
reported high levels of student engagement when students (11-12 years old) 
were using the platform and the Sense-it application. Students recorded sounds 
and lights in a shopping mall and conducted interviews with people who live 
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Figure 9.2: The Gardenwatch mission supported by the BBC and BTO. 


around the mall to identify whether living next to the mall affects daily life. In 
the classroom, students discussed data and wrote a proposal for improving the 
environment around the shopping mall. 

Another example of how supporting distance learning students with their 
understanding of Science has led to the development of new technologies espe- 
cially within the realms of Computer Supported Collaborative Learning is our 
work on shared simulations. Building on the work on learning from simula- 
tions in the conceptual change in science projects (see Chapter 8, and shared 
simulations (see e.g. the SharedArk project, Scanlon et al. 1993) our challenge 
has been to support learners at a distance working together on complex tasks. 


Case study 2: real time working together on simulations 

This second case study provides an interesting narrative around the need to 
reduce the isolation of students working remotely with a positive experience of 
lerning science together through working in pairs in real time on a computer 
simulation. This research came to fruition in 2006 by linking together, two 
software developments, which were built at the OU, known as BuddyFinder 
and SIMLINK. BuddySpace was the original application that was of interest for 
this particular Science learning innovation and it was developed as an Instant 
Messaging environment for community building see Vogiazoglou et al (2005). 
BuddySpace provided enhanced capabilities for users to manage and visualise 
the presence of colleagues and friends in collaborative working, gaming, mes- 
saging, and other contexts. Of particular interest to our Science project was 
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the role of graphical metaphors for presence, including maps,the BuddySpace 
team were also studying at this time the semantics of presence, in order to move 
beyond simple flags such as ‘online’ and ‘busy’ to include rich contextual and 
spatio-temporal information more appropriate to the user’s focus of activity. 

Buddy Finder was then developed, by the Knowledge Media Institute, to 
enable users to perform two different functions. First of all it allowed each user 
to input keywords describing their individual interests and/or skills. Secondly 
it allowed a user to search the user-defined keyword space to find another user 
or users that match a specified keyword. The idea being that students would 
describe themselves, for example, as knowledgeable about or interested in cli- 
mate change so that other students wishing to discuss this topic would be able 
to contact them via BuddySpace. Figure 9.3 shows the original BuddySpace 
messaging Interface while Figure 9.4 illustrates the BuddyFinder interface 
where students could see from the hotspots on the map, which students were 
actively online and also search to see if any were interested in working with 
them on the Global Warming topic see Figure 9.5 below 

SIMLINK was the second programme which was built to allow students to 
work together at a distance with the Global Warming simulation see Figure 9.5. 
The variables that could be changed in this simulation were those of; cloud 
cover, amount of ice and snow, albeado, aerosol content, water content, carbon 
dioxide content and the solar constant. See Figure 9.5 below. Students were 
linked together remotely using SIMLINK which was in essence a Java based 
downloadable plugin which formed part of the BuddySpace family of communi- 
cation tools. It allowed users at a distance to work on a joint simulation together. 
The users could view the same screen. This meant that when one student made 
a change to the simulation the other saw this change. In effect the pair working 
together were viewing identical representations on their monitors, as they would 
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Figure 9.3: BuddySpace messaging Interface. 
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Figure 9.5: SIMLINK with Global Warming. 
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as if they were working side by side. This was achieved by sending mouse click 
changes only from one partners application to the other. This avoided bandwith 
problems and time delays that ‘raw’ screen sharing would have entailed. 

Fourteen volunteers were able to work together in separate rooms using the 
BuddySpace and SIMLIMK software to solve problems about Global warm- 
ing together in a session which lasted up to 45 minutes. They quickly became 
engaged in the topic and cognitive change scores i.e. the difference between pre 
and post scores showed an increase. This trial illustrated how the system could 
be used for formative assessment purposes see (Whitelock 2008). The discourse 
revealed participants were encouraging each other to keep to the task and were 
happy to explore an unfamilaiar terrain together. The participants agreed that 
this combination of communication tools to present complex problem solving 
tasks would be a beneficial asset to OU students. The potential of BuddySpace 
was then recommended by the Open Content Initiative, to become integrated 
into Moodle. 

The main pedagogical driver for this research was to provide a screen shar- 
ing application that would assist with the development of complex problem 
solving formative assessments for students studying remotely. In this way a 
‘Predict, Look and Explaim’ modus operandi for collaborative work around 
a simulation that has well documented pedagogical benefits could be pro- 
vided for students studying alone and remotely. One of the major findings 
of this project is the creativity of staff, both academic and technical, to cre- 
ate formative e-assessments. The development process is time consuming 
and costly and fewer students benefit from these electronic formative assess- 
ments when they are an optional extra in the course. It was recommended that 
electronic formative assessment become a compulsory element of the course 
teams teaching learning materials. Learners have been shown to welcome the 
instant feedback afforded by electronic assessment, which can also be used by 
tutors to diagnose student misconceptions of a given topic. Designing forma- 
tive assessments around known misconceptions was a recommended outcome 
from this work. 


Conclusions 


The story of Citizen Inquiry combined insights from the Personal Inquiry (PI) 
project (described further in Chapter 8) and Citizen Science to develop a set of 
enabling Citizen Science technologies. It suggests new ways of capitalizing on 
the strength of Citizen Science for the benefits of each citizen and their com- 
munities. It is an interactive way of engaging learners with science and a way 
of making use of the pedagogical benefits of inquiry-based learning at a large 
scale. Overall, this journey aimed to bring science closer to the everyday life 
of learners and help them understand and appreciate its value by developing 
bridges between formal and informal education. It showcases how a successful 


148 Educational Visions 


secondary education project on structuring scientific investigations with the 
support of technology can be expanded to foster participation in research activi- 
ties in informal settings, with the support of technology and through communi- 
cation with others. Similarly, our recent collaboration with three natural history 
museums and two other universities (funded by the National Science Founda- 
tion (NSF), Wellcome Trust and Economic Social Research Council (ESRC)) 
aims to understand what young people currently learn from their participation 
in Citizen Science programs led by museums, and to redesign these to effec- 
tively scaffold informal learning experiences (LEARN Citizen Science, 2017). 
These activities emphasise the importance of promoting a continuation of learn- 
ing across settings, supporting self-regulated learning with the help of technol- 
ogy and social interactions, and promoting lifelong learning inspiration. 

Our work on collaboration at a distance supported by technology described 
here, but also in other accounts of our experiments on collaborative problem 
solving at a distance using shared simulations (see e.g. Scanlon et al. 2005) also 
demonstrated the commitment to the judicious use of technology to overcome 
the challenges of learning science at a distance. These two themes are facing 
the contemporary challenge of assessing learning appropriately and effectively. 

Our research engagement with science projects and activities over the years 
has been shown to be challenging and has required flexibility to accommodate 
difficulties, a strong shared vision and persistent intent. Our major challenge 
has been the lack of funding available that threatened the implementation of 
our vision of “engaging learners enthusiastically with science”. Persistent intent 
and ongoing communication amongst the research team contributed signifi- 
cantly in identifying solutions and pursuing, slowly but steadily our vision. 
Reflecting on the last 40 years, we could argue that we did manage to engage 
learners with science through innovative pedagogies such as Citizen Inquiry, 
learning from simulations and a range of web-based and mobile technologies. 
What now requires further work is to ensure that learners engage “enthusiasti- 
cally” with science; that they are satisfied with their participation in science 
activities and see learning as fun and enjoyable. Going forward, we need to 
address the challenges of evidencing learning in informal settings, or connect- 
ing learning across settings, and ensuring that this learning is enjoyable and 
engaging. In pursuit of this we should aim to create learning environments that 
allow for playful exploration and experimentation, promote making and recov- 
ery from mistakes, and expand over and above formal education, to profes- 
sional and lifelong learning (Ferguson et al., 2019). 
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CHAPTER 10 


Visions for the Future of 
Educational Technology 


Mike Sharples 


Educational Technology is in a period of exciting change, with new technologies 
such as augmented and virtual reality, new techniques to analyse student data, and 
new pedagogies for learning online at large scale. Universities are entering part- 
nerships with publishers and startup companies to develop teaching and learning 
online. As the Computer and Learning research group (CALRG) celebrates its 40th 
anniversary, The Open University (OU) faces challenges and opportunities. The 
challenges are to find answers to three big questions. How can providers of online 
courses develop sustainable business models? How can institutions work together 
to develop courses that attract substantial numbers of fee-paying students and offer 
transferrable credit? How can course designers offer education that is both engaging 
and effective? The opportunities include developing new partnerships though the 
FutureLearn company to offer professional development courses with transferable 
credit, exploring inquiry learning at scale with the nQuire platform in collaboration 
with the BBC, and developing mobile technologies that promote broad and deep 
access to learning. A promising future research agenda is to examine how new edu- 
cational technology can combine personalized with social learning. A lesson from 
40 years of CALRG is that that successful computer-assisted learning involves not a 
series of exciting prototypes and quick fixes, but a sustained programme of research 
into the science of learning and the design of effective interventions. 
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Introduction 


The introductory chapter to this book shows an extract from a speech by Lord 
Crowther, the first Chancellor of The Open University, where he refers to the rev- 
olution in communications enabled by computers. Since that speech in 1969, the 
world has undergone a further technology revolution brought by mobile commu- 
nications devices: smartphones, tablet computers and wearable communicators. 
Equally important is the revolution in education. In 1969, university students 
listened to lectures, wrote course notes, attended seminars, and sat in exam 
halls. Many still do. Yet people of all ages and nationalities now learn online. 
They look up Wikipedia to understand Bitcoin, watch a YouTube video to find 
out how to bleed a radiator, go to a blog to find a recipe for lasagne, and browse 
TripAdvisor to plan a holiday. Also, since 2012, Massive Open Online Courses 
(MOOCs) have allowed anyone with a fast internet connection to study a 
course in, for example, Mathematics, Machine Learning, or Mindfulness. 


The MOOC phenomenon 


Many academics at The Open University (OU) were initially blasé about the 
MOOC phenomenon. Since 1999, we have had our own Open2.net site’, 
rebranded in 2006 as OpenLearn’. It provides hundreds of hours of free edu- 
cational content, in collaboration with the BBC. However, in late 2012 the OU 
made a decision to form the FutureLearn company and build a new platform? 
to offer free online courses from leading universities worldwide. This generated 
challenges and opportunities. 


Project Kyloe 


The main challenge was to build a consortium of universities willing to develop 
free courses, for people from all nationalities with little or no experience of 
online learning. The opportunity was to develop a new platform that would 
engage people in sustained, effective, self-managed learning. 

In early December 2012, a small group of educational technology experts at 
the OU were asked to comment on a set of features for Project Kyloe’, the code 
name for what was to become FutureLearn. Should the platform have recom- 
mendation features, email alerts, ebooks or pdfs, presenters or guest tutors? 


1 https://www.open.edu/openlearn/about-openlearn/frequently-asked-questions/ 
looking-open2net 

2 https://www.open.edu/openlearn/ 

3 www.futurelearn.com 

4 Kyloe is a type of Scottish Highland cattle: Kyloe, cattle, moo, MOOC. 
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Asking experts to propose a list of features is exactly the wrong way to design 
a new platform for learning. The right way is to start from the pedagogy. What 
types of teaching, learning and assessment should be supported? How will peo- 
ple from differing cultures, languages and educational backgrounds be helped 
to engage and learn? Fortunately, at the end of December 2012, the then Vice 
Chancellor, Martin Bean, convened a meeting to develop a vision for the new 
MOOC platform and establish a small team to work with the newly-formed 
FutureLearn company on pedagogy-informed design of the platform. 


Designing FutureLearn 


Members of CALRG were prominent in that team, with Sharples as Academic 
Lead. Together with the software developers, they based design of FutureLearn 
on a pedagogy of learning as conversation". Conversation is a fundamental 
process of learning. We converse with colleagues and teachers to share knowl- 
edge and coordinate actions. We converse with ourselves to reflect on experi- 
ence. Conversation can also improve with scale: the more people that take part, 
the richer and more diverse is the discussion. In FutureLearn, each piece of 
teaching is linked with a conversation amongst the learners. Conversation for 
learning has been the guiding principle for designing new features such as peer 
assessment and online study groups. 

In May 2015, 270,00 people started the FutureLearn course ‘Understand- 
ing IELTS’ from the British Council. The biggest-ever online MOOC course, 
it attracted learners from 190 countries notably in the Middle East and Eastern 
Europe. For many, it was their first experience of online learning. 25% accessed 
the courses on mobile devices. The first video in that course (asking participants 
how they feel about taking exams) attracted 65,000 comments. By the time the 
course had ended, over 35% of the participants had contributed to the online 
discussions and many more had learned from viewing the peer contributions 
alongside educator-designed content. This and subsequent courses have shown 
how well-conceived social learning can be a basis for open education at scale. 


FutureLearn Academic Network 


Perhaps the greatest opportunity afforded by FutureLearn has been for aca- 
demics and educational technologists from 120 institutions to explore new 
ways to teach online. The FutureLearn Academic Network (FLAN) was set up 


5 Conversation Theory was developed by Gordon Pask and extended by Diana Lau- 
rillard, both of whom worked with The Open University. Laurillard was a former 
Pro-Vice-Chancellor (Technology Development). Pask obtained the first Open Uni- 
versity DSc. 
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in 2013 to connect academics and research students based in FutureLearn part- 
ner institutions. FLAN is coordinated by Eileen Scanlon and Rebecca Fergu- 
son from CALRG. Through quarterly meetings, comparative research studies, 
joint research bids and collaborative publications, the network has examined 
successful ways to teach, learn and assess online. Two reports, authored by 
members of CALRG, surveyed 66 publications on MOOCs from The Open 
University (Ferguson, Coughlan & Herodotou, 2016) and then 109 publications 
by FutureLearn partners (Ferguson, Coughlan, Herodotou & Scanlon, 2017). 
The two reports identify priority areas for universities investing in MOOCs, 
including: develop a strategic approach to learning at scale, identify and share 
learning designs, support discussion more effectively and widen access. 

The pedagogy of learning through conversation at scale has also posed new 
questions for educators. What makes a good question to prompt discussion? 
What should be the role of educators in facilitating conversation - should 
they ask open questions, offer hints, answer queries, encourage peer discus- 
sion, lubricate social interactions? Can learners be trained and supported 
to act as peer facilitators? How should courses be designed to engage and 
retain students? 


A journey from MOOCs to micro-credentials 


FutureLearn, in common with other MOOC platforms, is now on a long-term 
journey away from a focus on free courses for leisure learners, towards accred- 
ited programmes for professional development and lifelong learning. Figure 1 
shows the trends, from 2000 to 2020, of open and distance education. In the 
early 2000s, some universities including The Open University made educational 
resources such as course notes and recordings of lectures free to browse online. 

In the first experiments with MOOCs, from 2008 onwards, learners con- 
structed free online resources into personalized courses and discussed their 
learning with other participants. In 2012, the major MOOC platforms of Cour- 
sera and EdX were established, followed in 2013 by FutureLearn. The courses 
that run on these platforms, along with others including OpenLearn, have 
generated rich data on student learning. The fields of social learning analytics, 
predictive analytics and analytics for learning have sprung up to inform new 
methods of teaching and learning at scale. 

A combination of business imperatives (MOOC companies have belatedly 
realised that it’s hard to sustain a business based on free courses), entrepre- 
neurship, and greater understanding of the needs and profiles of adult learn- 
ers, have resulted in clusters of courses, dubbed ‘nano-degrees. Each of these 
nano-degrees provides a credential certified by the providing institution. Com- 
bine these clusters, sometimes from multiple providers, and you have a hybrid 
degree course. Merge them with campus teaching and you get blended courses 
that can be taken on campus, online, or in combination. 
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Figure 10.1: Trends for open and distance education from 2000 to 2020. 


This complex net of partnerships, providers and pedagogies is still under 
development, not least at The Open University. A postgraduate degree in 
Online and Distance Education has been developed by faculty associated with 
CALRG and now runs on the FutureLearn platform. This degree is both a way 
to apprentice students into e-learning and a means to research the delivery of 
accredited courses on a MOOC platform. 


The adult learning dilemma 


Such courses expose a central dilemma of adult learning: what students 
like most is generally not what is best for them. In 2016, Rienties and Toe- 
tenel, from CALRG, published two papers that analysed student satisfaction, 
retention and performance for over 150 degree modules offered by the OU 
(Rienties & Toetenel, 2016; Toetenel & Rienties, 2016). To develop an OU 
undergraduate module, the course team follow a process of learning design 
that involves predicting the percentage of different types of student learn- 
ing: assimilating delivered content, finding information, communicating 
with other students, producing assignments, experiencing, interacting, and 
taking assessments. 

When the module runs, the university gets data on student satisfaction, reten- 
tion and exam performance. Thus, for each OU module, we can investigate 
what type of course design produces what outcomes. In brief, students prefer 
modules with plenty of delivered content (videos, texts) and some interaction. 
But the modules best at retaining students are those with communicative and 
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collaborative learning. And student exam performance is worst on those mod- 
ules that are heavy on delivered content. 

The future of distance education rests on finding answers to three big ques- 
tions. How can providers of online courses develop sustainable business 
models? How can institutions work together to develop courses that attract 
substantial numbers of fee-paying students and offer transferrable credit? How 
can course designers overcome the adult learning dilemma to offer education 
that is both engaging and effective? For The Open University, as one of the 
largest distance learning universities in Europe, addressing these questions is 
central to its strategy and future direction. 


Inquiry learning at scale 


FutureLearn has shown that a platform based on a pedagogy of learning as 
conversation can be both engaging and effective at massive scale. What other 
methods of teaching and learning can run at scale? Since 2007, CALRG has 
designed a series of technologies, collectively named nQuire, to investigate 
inquiry learning, initially in classrooms then from 2013 for self-directed learn- 
ing online. The approach of pedagogy-led design is the same as for FutureLearn: 
start from theory and practice of inquiry learning; let this inform design of a 
demonstrator system that is tested with learners; apply findings from the sys- 
tem in use to inform both design of the next version and to refine the pedagogy. 


Personally meaningful inquiry 


nQuire has gone through three main phases, each offering insights into inquiry 
learning with technology. The first was to explore ‘personally meaningful 
inquiry. School students investigated topics that had personal significance, 
such as ‘Are animals in cities affected by pollution?’ and “How noisy is my class- 
room?’ Each student had a computer-based toolkit to guide an entire inquiry 
process that connected structured learning in the classroom with discovery 
and data collection at home or outdoors. A visual map of the inquiry process, 
enacted on a portable computer, was successful in guiding students. However, 
this schools’ version of nQuire placed demands on the teacher to orchestrate 
the process, particularly for the classroom activity that integrated data collected 
by all the students into a satisfying conclusion. 


Citizen inquiry 
The second phase, described in Chapter 9, was to explore how inquiry learn- 


ing could be managed online, without the guidance of a classroom teacher. 
The structured inquiry process from the first phase proved too complex and 
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tedious for self-managed inquiries, so we have explored how to implement a 
new pedagogy of ‘citizen inquiry’ (Herodotou, Sharples & Scanlon, 2017). This 
fuses the mass participation of citizen science with the question-led investiga- 
tion of inquiry learning. In effect, it flips the roles of citizen science. Instead of a 
scientist designing an investigation to which members of the public contribute, 
in citizen inquiry any person or group can design an inquiry and then recruit 
other people, including scientists, to help with carrying it out. The key is to keep 
each inquiry clear and focused, with an initiating ‘big question; a structured 
activity to investigate it, data that can be collected by members of the public 
on mobile devices, a way to publish results on the platform, and a conversation 
between participants to provoke interest and guide the unfolding investigation. 

The nQuire-it platform was open to anyone to develop a new inquiry, called 
a ‘mission. An authoring tool on nQuire-it assisted in designing the mission. 
Once the mission had been built, it was published on the site for anyone to con- 
tribute. Each contribution was visible and, as with FutureLearn, each mission 
and contribution had a linked discussion. Over the five years that the platform 
ran, some 150 inquiry missions were developed, ranging from an investigation 
of noise levels in school classrooms to observations of the impact of flooding on 
homes and roads in Vietnam’s Mekong Delta (Figure 10.2). 


Figure 10.2: A contribution to an nQuire-it mission to log and discuss flooding 
in Vietnam. 
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Although nQuire-it demonstrated that people worldwide could design and 
run investigations, most missions were short-lived with few contributions. 
A study by Aristeidou with amateur meteorologists (Aristeidou, Scanlon & 
Sharples, 2017) found that extrinsic factors, such as a well-designed site and 
thought-provoking questions, attracted and activated participants, but intrin- 
sic factors such as personal interest in the topic and support from the com- 
munity crucially determined whether the community of inquiry developed 
and sustained. 


Inquiry learning at scale 


The new nQuire platform’ is a collaboration with the BBC. The aim is to demon- 
strate scale and sustainability in learning by inquiry. Promotion through BBC 
broadcast media provides the initial recruitment (a mission on nQuire to sur- 
vey UK gardens has attracted 230,381 responses), but to be successful nQuire 
must address three challenges: engage participants in valid and meaningful 
scientific inquiry; provide reward for taking part; enable individuals, groups 
and organisations to design and run missions on the platform. Some meth- 
ods we are exploring to meet these challenges include: framing each mission 
within a scientific process that includes informed consent and ethical scrutiny; 
offering intrinsic rewards through personalized feedback; giving powerful yet 
easy-to-use tools to author new investigations; and providing a guided process 
to design, preview, pilot and launch a mission. 

The nQuire project offers an object lesson in developing education technol- 
ogy beyond prototypes (Scanlon, et al., 2013). The persistence has lasted over 
twelve years and four major versions of the platform. Each iteration of pedagogy 
and technology has produced a site for bricolage through playful experiment. 
For example, the Noise Map mission on nQuire-it started as a demonstration of 
how a mobile phone could capture sound data to the platform, then was taken 
up by a teacher in Argentina who used it with students to explore environ- 
mental noise, then it spread to schools in Hong Kong, Taiwan, China and New 
Zealand where they compared noise levels in classrooms, labs and school cafes: 


“Our MindLab Manurewa recorded a sound level of 44 to 68dB. While 
this seemed somewhat reasonable, I think this was a little loud for 
groupwork indoors. At a high of 68dB we struggled to hear each other 
or follow the conversation.” ejenkins@ormiston.school.nz 


Each iteration of the nQuire project has produced evidence of the value to learn- 
ers of engaging in inquiry learning, both to investigate personally-meaningful 


6 www.nquire.org.uk 
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issues and to understand how to be a scientist. It has also raised issues - such 
as teacher orchestration, self-managed inquiry, and inquiry at scale — that have 
been examined by successive iterations. However, if this suggests a smooth 
path of design-based research, it certainly does not feel that way to the research 
team. Each version of the project has involved debate about the value and direc- 
tion of the research and a continual struggle to get funding. 

Inquiry, like conversation, is a fundamental process of learning. Through 
inquiry learning students learn to pose thoughtful questions, make sense of 
information, and learn about the world around them. They develop the skills 
and attitudes needed to be self-directed, lifelong learners (National Library of 
New Zealand, n.d.). Citizen inquiry learning that is online, open and scalable 
offers new opportunities for people to learn by investigating themselves and 
their environment. 


Mobile and accessible learning 


Personal investigations require mobile technologies to record and share data. 
Members of CALRG have been active since the early 2000s in development 
of mobile technologies for learning. Sharples held the first international con- 
ference on mobile and contextual learning, to become the mLearn conference 
series, and both Sharples and Kukulska-Hulme have served as Presidents of the 
International Association for Mobile Learning’. 


Learning and context 


The large European MOBILearn project, which ran from 2002-2004, devel- 
oped an architecture for mobile learning. As with FutureLearn and nQuire, the 
focus was not on the technology alone, but the combination of pedagogy and 
technology. One important legacy of MOBILearn is a theory of mobile learning 
as a contextualised practice (Sharples, Taylor & Vavoula, 2016). 

As learners, we are always immersed in a context. For traditional education, 
this is the classroom, managed by a teacher and mediated by familiar tools such 
as blackboards and textbooks. When education is taken beyond the classroom, 
the context becomes more fluid and unpredictable. As well as being in a con- 
text, as learners we also create context out of our available resources of location, 
technology and social setting. For example, a family standing before an exhibit 
in a museum is creating a context for learning out of the exhibit and its label- 
ling, the route through the museum to reach that exhibit, existing knowledge 


7 https://iamlearn.org/ 
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brought by each family member, mobile devices including handheld museum 
guides and mobile phones, and the conversation amongst the family members. 

Since people have a diversity of needs and cultures, own a range of tech- 
nologies, and move through varying locations and social engagements, it fol- 
lows that researching mobile learning is complex and challenging. To design 
mobile technologies involves either adapting them to rapidly changing contexts 
or providing a generic aid to learning that will offer useful learning despite 
context (Sharples, 2015). To evaluate the effectiveness of mobile learning 
requires new methods for understanding how knowledge is created within and 
across contexts. 


Mobile support for migrants 


Of all the contexts for mobile learning, perhaps the most difficult to research 
and design involve support for migrants. Immigrants to a country bring their 
own knowledge, language, technologies, expectations and concerns to a new 
setting. Just providing migrants with a mobile visitor guide in their own lan- 
guage in by no means sufficient. They have needs to gain work, find friends, 
understand cultural norms and expectations, and learn how to survive and 
prosper in the new environment. CALRG, led by Agnes Kukulska-Hulme, 
has had a central role in the European MASELTOV project to support immi- 
grants in cities through mobile technologies. Its aim has been to understand 
the changing contexts of the immigrants as they go about their daily lives and 
how combinations of mobile technology and peer support can give help when 
needed (Kukulska-Hulme, et al., 2015). 


Accessible learning 


Viewing learning as a mobile and contextualised activity prompts us to rethink 
accessibility. As Chapter 4 discusses, allowing people with a broad variety of 
abilities and disabilities to enter online learning is necessary but not sufficient. 
We must also support them to stay and learn. Each person has a different con- 
text for learning - with unique needs, barriers, resources, culture, and social 
network. This context frames how that person understands what it means to 
learn, what will be gained from engaging in education, and how the learning 
activity will progress. The implication is that we must look for new ways to 
support the resourcefulness of students from their contexts, not just provide 
our resources. 

As an example, members of CALRG have been exploring the value of predic- 
tive analytics. Computational techniques can analyse the online interactions 
of students on a course and predict, with high accuracy, which students are 
at risk of failing. What then? One use of such methods is to alert teachers to 
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poorly-performing students. Another is to let the students themselves know 
what they could do to get back on track - such as join a discussion forum or 
read a supplementary text. However, this is analytics seen from the perspective 
of the course provider. To the student, being given yet more resources is unlikely 
to help if they are already overloaded, or if they view learning as a process of 
trying to digest everything they are given. Understanding students’ context, 
history and culture may contribute more to effective learning than diagnosing 
failure — but that is hard to program into software. A better approach may be to 
offer analytics that empower teachers to understand not just which students are 
at risk, but the contextual factors involved (Herodotou, et al., 2019). 


Merging personal and social learning: a research agenda 


It is an exciting time for educational technology. Theres a plethora of new tools 
for learning: virtual and augmented reality, chatbots, predictive analytics, per- 
sonalized learning systems. The Bill and Melinda Gates Foundation has teamed. 
up with the Zuckerberg Initiative (founded by Facebook CEO, Mark Zucker- 
berg) to promote and invest in personalized technology for classroom instruc- 
tion (New Profit, 2017). 

However, technology alone will not transform education. An analysis of 40 
years of research into the impact of educational technology on educational 
performance shows only a small to moderate effect size of 0.33 (Tamin, et al., 
2011). The successes in computer-assisted learning come from understanding 
how to use technology effectively in the classroom and online. Future research 
must explore good combinations of technology and pedagogy. For example, 
a RAND study of personalized learning in schools showed that it could be 
effective, but only if students learn in groups, the classroom is re-designed to 
accommodate the new way of learning, and the students are given opportu- 
nities to discuss their performance with the teacher (Pane, Steiner, Baird & 
Hamilton, 2015). 

A promising research agenda for educational technology is to examine how 
personalized and social learning can fit together. Personalized learning offers 
content and activity that is matched to the needs, abilities and context of each 
learner. It can drive mastery learning where the student continues with a topic 
until it is well understood, and cognitive tutoring that diagnoses each student's 
knowledge and gives remedial help to correct misconceptions. 

Social learning is a great success of educational innovation (Johnson & 
Johnson, 2009). When students cooperate in small groups of between four 
and eight people, this results in greater creativity and better outcomes than 
working alone. Over the past 40 years, hundreds of studies in labs, classrooms 
and online, have uncovered conditions for successful cooperative learning. 
For groups to work well, they need to have shared goals, each person should 
know how and when to contribute, and everyone should make an appropriate 
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contribution. They should share rewards such as group marks in a fair way, and 
members of a group should all have opportunities to reflect on progress and to 
discuss contributions. For many students, learning in groups is not a natural 
process, and they need to learn how to cooperate by arguing constructively and 
resolving conflicts. The key phrase is positive interdependence - everyone sees 
the benefits of learning together and works to achieve the group’s goals. Social 
learning platforms such as FutureLearn are starting to show how positive inter- 
dependence can enhance learning online at large scale. 

How can personalized and social learning be made to work together, so that 
they compliment rather than conflict? It means designing learning environ- 
ments that encourage students to examine their personal learning goals and 
work to achieve mastery of a subject, not alone but alongside others with simi- 
lar aims and contexts. Successful learning environments of the future will be 
based on a deep understanding of the science of learning, support students to 
set and meet their goals, offer a combination of personalized tuition and social 
learning, harness predictive analytics to assist teachers and students, and pro- 
vide a delightful experience. 


Ethical EdTech 


Yet even an environment for successful learning is not enough, if it fails to reach 
the standards expected of ethical research and development. Too many edu- 
cational technology studies in the past have treated students as if they were 
subjects in a laboratory experiment. That is no longer acceptable. The new 
direction is not to claim that the outcomes of an educational intervention jus- 
tify the means, nor to rely on an ethical review board to police educational 
research, but for the researchers themselves to engage actively in a process of 
ethical research design (Head, 2018). 

The first consideration is whether it is ethical for any piece of educational 
research to take place at all. Researchers should engage in questioning the 
assumptions of their research from the outset. Part of developing as an educa- 
tional researcher is learning how to work with participants, to be sensitive to 
their needs and contexts, and to address and resolve ethical dilemmas. “Becom- 
ing an ethical educational researcher, then, is a matter of pedagogy.” (Head, 
2018, p.11). The CALRG is contributing to that pedagogy of ethics for fields 
that include mobile learning (Lalley, et al., 2012) and AI and education (Hol- 
mes, Iniesto, Sharples & Scanlon, 2019). 

It is not possible here to summarise the rich ongoing discussions about eth- 
ics for educational research. Instead this chapter ends with some provocative 
guidelines framed as a mnemonic: MISSION. 


Multiple media, devices, partners 
Independent verification 
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Secure environment 
Support for learners 
Inquiry process 
Open access 
aNalytics for learning 


Educational technology requires understanding the multiple media and devices 
that people use in their everyday lives and designing new ways to augment their 
learning across contexts. It also involves multiple partners as design inform- 
ants, including learners, teachers and policy makers. 

As design of an educational technology progresses, there is increasing need 
to bring in independent verification of the educational need, the user experi- 
ence, the validity of the learning and its ethical soundness. For CALRG pro- 
jects, this has included expert testing of early prototypes, recruiting teachers 
as experts in educational effectiveness and curriculum relevance, and commis- 
sioning independent reviews of data security. 

A secure environment for learning covers not only ensuring security of data 
produced by learners, but also providing safe and enjoyable places to learn 
online. For example, the FutureLearn and nQuire sites provide ways for users 
to report inappropriate comments, checked by moderators. 

Support for learners starts from their first engagement with the educational 
technology. For our projects, we put great effort into designing the ‘first five 
minutes experience’ so learners know how to take part, what to expect, and 
what to do first. On nQuire, users can view content without registering for the 
platform, so they know what to expect before providing personal data. Support 
for learners can extend to recruiting expert and peer facilitators and embed- 
ding effective pedagogies into the learning experience, such as formative test- 
ing with immediate feedback. 

An inquiry process is central to active learning. It involves learners setting 
personal goals and asking questions to themselves that require investigation 
and reflection. The teacher becomes a partner in the learning process, guiding 
students to create knowledge. Good teachers and researchers inquire into their 
own practices and share knowledge of what works. 

For ethical education, open access should be the norm, not only to enter 
education but to profit from the full richness of the experience. That means 
designing for cultural, physical and mental diversity and providing ways for 
like-minded students to share their knowledge and experience. It also means 
giving students access to the process and results of their unfolding study, 
through techniques such as dynamic knowledge maps, skill charts and open 
learner models (Bull & Kay, 2010). 

Learning online creates a rich seam of data that can be mined to show progress 
and performance. An ethical approach to educational technology harnesses 
that data to provide analytics for learning. This could take the form of predictive 
analytics to guide students in what to do next based on their performance, or it 
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could assist teachers and course designers to understand which topics students 
find difficult and how to improve the course quality and access. 


CALRG at 40 


As the Beyond Prototypes framework indicates (Scanlon, et al., 2013), tech- 
nology enhanced learning is a complex system of technologies and practices, 
developed and embedded over many years. We are fortunate to be part of a 
research group that has prospered for 40 years and is still thriving. Some themes 
that influenced the formation of CALRG, such as designing and testing open 
and accessible educational technologies, are as important now as then. Some 
themes have come to prominence more recently, such as developing learn- 
ing environments that are scalable and sustainable. And some themes set an 
agenda for future research and development, including analytics for learning 
and how to combine personalized with social learning online. Over the years, 
PhD students have made a major contribution to CALRG, opening new areas 
of research and bringing their personal and cultural perspectives. So too have 
the system developers and programmers - they have built the technologies and 
platforms, sometimes through many versions, that support the learning and 
test the theories. 

We now know for certain that successful computer-assisted learning involves 
not a series of exciting prototypes and quick fixes, but a sustained programme 
of research into the science of learning and the design of effective interven- 
tions. Continued support for research and development in education technol- 
ogy is essential for the next generation of students to benefit from the current 
advances in educational technology and pedagogy. 
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